Core Concepts in Empirical Economics and Causal Inference, Part 2

7. Randomized Controlled Trials, or RCTs
8. Natural Experiments and Quasi-Experimental Design
9. Identification Strategy
10. Instrumental Variables
11. Difference-in-Differences
12. Regression Discontinuity

Introduction to Part 2

This document is a continuation of the empirical concepts series. Part 1 developed the foundations of causal inference: correlation versus causation, counterfactual thinking, potential outcomes, DAGs, exogeneity, endogeneity, omitted variable bias, confounding, and selection bias. Part 2 begins the transition from causal problems to causal research designs. The central question now becomes:

Given that causal effects require credible counterfactuals, what empirical designs can help us construct them?

The first and most important benchmark design is the randomized controlled trial. Randomized controlled trials are not the only valid approach to causal inference, and they are not always feasible or sufficient. But they provide the cleanest conceptual model for understanding what a credible causal comparison requires. An RCT shows, in its simplest form, what empirical researchers are usually trying to approximate: a treated group and a control group that differ systematically only in treatment status.

7. Randomized Controlled Trials, or RCTs

7.1 Why randomized controlled trials matter

A randomized controlled trial, or RCT, is a research design in which units are randomly assigned to treatment or control. The units may be individuals, households, classrooms, schools, firms, villages, hospitals, neighborhoods, or other entities. The purpose of random assignment is to solve the central problem of causal inference: the missing counterfactual. For each unit $i$, define two potential outcomes: $Y_i(1)$ and $Y_i(0)$ where:

$Y_i(1)$ is the outcome unit $i$ would have under treatment,
$Y_i(0)$ is the outcome unit $i$ would have under control or no treatment.

The individual treatment effect is: $\tau_i = Y_i(1)-Y_i(0)$ But for each unit, only one potential outcome is observed. If the unit receives treatment, we observe $Y_i(1)$ but not $Y_i(0)$. If the unit does not receive treatment, we observe $Y_i(0)$ but not $Y_i(1)$. An RCT addresses this problem by assigning treatment randomly. Randomization makes the treated and control groups comparable in expectation. The control group is used to estimate what would have happened to the treated group if it had not received treatment. This is why RCTs are often treated as the benchmark for causal inference. They create a comparison group by design rather than relying only on statistical adjustment after the fact. The core idea is simple:

If treatment is randomly assigned, then treatment status is unrelated to both observed and unobserved determinants of potential outcomes, except by chance.

This does not mean every treated unit is identical to every control unit. It means that, before treatment, the two groups are drawn from the same underlying population by a random process. Therefore, systematic differences in average outcomes after treatment can be attributed to the treatment, subject to sampling error and design validity.

7.2 Treatment, control, and random assignment

An RCT requires at least three elements:

A treatment condition,
A control condition,
A random assignment mechanism.

The treatment condition is the intervention being evaluated. Examples include:

receiving job training,
being offered tutoring,
receiving a cash transfer,
being assigned a new curriculum,
receiving a medication,
receiving information about a program,
being exposed to a pricing change,
being assigned to a new management practice.

The control condition is the comparison state. It may be:

no treatment,
business as usual,
a placebo,
delayed treatment,
a different version of treatment,
a lower-intensity treatment,
the existing policy regime.

The random assignment mechanism determines which units receive treatment. For a binary treatment, let: $D_i \in \{0,1\}$ where:

$D_i=1$ means unit $i$ is assigned to treatment,
$D_i=0$ means unit $i$ is assigned to control.

In the simplest RCT, treatment assignment is generated by a random device such as a coin flip, lottery, random number generator, or randomized algorithm. If assignment is truly random, then $D_i \perp (Y_i(1),Y_i(0))$ This means treatment assignment is independent of potential outcomes. In words:

Whether a unit is assigned to treatment is unrelated to what its outcome would have been under treatment or control.

This condition is what makes the control group a credible counterfactual for the treatment group.

7.3 The basic RCT estimand

The average treatment effect, or ATE, is: $ATE = \mathbb{E}[Y_i(1)-Y_i(0)]$ Using linearity of expectation: $ATE = \mathbb{E}[Y_i(1)] - \mathbb{E}[Y_i(0)]$ In an RCT with perfect compliance and no attrition, random assignment identifies the ATE because: $\mathbb{E}[Y_i(1)] = \mathbb{E}[Y_i \mid D_i=1]$ and $\mathbb{E}[Y_i(0)] = \mathbb{E}[Y_i \mid D_i=0]$ Therefore: $ATE = \mathbb{E}[Y_i \mid D_i=1] - \mathbb{E}[Y_i \mid D_i=0]$ The sample estimator is usually a difference in sample means: $\widehat{ATE} = \bar{Y}_1 - \bar{Y}_0$ where $\bar{Y}_1 = \frac{1}{n_1}\sum_{i:D_i=1}Y_i$ and $\bar{Y}_0 = \frac{1}{n_0}\sum_{i:D_i=0}Y_i$ Here, $n_1$ is the number of treated units and $n_0$ is the number of control units. This estimator is simple, but its simplicity is a strength. The causal credibility comes from the design, not from a complicated statistical model.

7.4 Why randomization solves confounding

Confounding occurs when treatment status is related to other causes of the outcome. For example, suppose we want to estimate the effect of job training on earnings. In an observational study, people who participate in training may differ from nonparticipants in motivation, prior work experience, education, local labor market conditions, and social networks. A DAG might look like this: $Motivation \rightarrow Training$ and $Motivation \rightarrow Earnings$ with $Training \rightarrow Earnings$ The backdoor path is: $Training \leftarrow Motivation \rightarrow Earnings$ If motivation is unobserved, a regression of earnings on training may be biased. Random assignment breaks the link between motivation and treatment assignment. If treatment is assigned by lottery, then motivated and less motivated workers are, in expectation, equally likely to be assigned to treatment. Under random assignment: $D_i \perp Motivation_i$ and more generally: $D_i \perp (Y_i(1),Y_i(0))$ This means randomization balances both observed and unobserved determinants of outcomes in expectation. This is the central advantage of RCTs over many observational comparisons. Researchers do not need to observe every confounder if treatment assignment is genuinely random.

7.5 Randomization in expectation versus balance in a sample

Randomization guarantees balance in expectation, not necessarily exact balance in any particular sample. Suppose a job training RCT randomly assigns 1,000 workers to treatment and 1,000 workers to control. Because of randomization, we expect the two groups to have similar average education, age, prior earnings, motivation, health, and work experience. But randomization does not guarantee exact equality. By chance, the treatment group may have slightly higher prior earnings or slightly more college-educated workers. This is called random imbalance. Random imbalance is not the same as selection bias. Selection bias occurs when treatment assignment is systematically related to potential outcomes. Random imbalance occurs by chance and becomes smaller as sample size grows. Researchers often report baseline balance tables to check whether treatment and control groups look similar before treatment. A balance table might compare groups on pre-treatment variables such as:

age,
gender,
race or ethnicity,
education,
baseline income,
prior employment,
baseline test scores,
health status,
household size,
location.

A typical balance regression is: $X_i = \alpha + \pi D_i + v_i$ where $X_i$ is a pre-treatment covariate. If randomization worked, estimates of $\pi$ should usually be close to zero, apart from chance variation. However, balance tests must be interpreted carefully. A few statistically significant differences can appear by chance if many covariates are tested. Also, failure to reject imbalance does not prove perfect balance. Balance tables are diagnostic tools, not the foundation of randomization. The foundation is the assignment procedure itself.

7.6 Estimation using regression

Although an RCT can be analyzed with a simple difference in means, researchers often use regression. The simplest regression is: $Y_i = \alpha + \beta D_i + u_i$ In this regression, $\beta$ equals the difference in mean outcomes between the treatment and control groups. With covariates, the regression becomes: $Y_i = \alpha + \beta D_i + X_i'\gamma + u_i$ where $X_i$ is a vector of pre-treatment covariates. In a valid RCT, covariates are not required for identification. Randomization already identifies the causal effect. But covariates can improve precision if they explain variation in outcomes. For example, if baseline test scores strongly predict post-treatment test scores, controlling for baseline scores can reduce residual variance and produce more precise estimates. Important rule:

In an RCT, control variables should generally be pre-treatment variables.

Controlling for post-treatment variables can introduce bias because such variables may be affected by treatment. For example, suppose tutoring affects study time, and study time affects test scores: $Tutoring \rightarrow Study\ Time \rightarrow Test\ Scores$ If the goal is to estimate the total effect of tutoring, controlling for study time would block part of the treatment effect.

7.7 Intent-to-treat effects

In many experiments, there is a difference between being assigned to treatment and actually receiving treatment. Let: $Z_i \in \{0,1\}$ represent assignment to treatment, and let: $D_i \in \{0,1\}$ represent actual treatment receipt. In an ideal experiment, $Z_i=D_i$ for every unit. But in many real experiments, some people assigned to treatment do not take it up, and some people assigned to control access treatment anyway. The intent-to-treat effect, or ITT, is the effect of assignment to treatment: $ITT = \mathbb{E}[Y_i \mid Z_i=1] - \mathbb{E}[Y_i \mid Z_i=0]$ The ITT answers:

What is the effect of being offered or assigned to the intervention?

This is often the most policy-relevant effect when policymakers can offer a program but cannot force people to participate. For example, a government may offer job training to randomly selected workers. Some workers may not attend. The ITT estimates the effect of offering training, not the effect of actually attending training. The ITT is valuable because assignment $Z_i$ remains randomized even if actual participation $D_i$ is not.

7.8 Treatment-on-the-treated and noncompliance

Researchers may also want to know the effect of actually receiving treatment. This is sometimes called the treatment-on-the-treated effect, or TOT. A simple but often imperfect formula is: $TOT \approx \frac{ITT}{Takeup\ Rate\ Difference}$ More formally, if assignment affects outcomes only through treatment receipt, then random assignment can be used as an instrument for actual treatment. The first stage is: $D_i = \pi_0 + \pi_1 Z_i + v_i$ The reduced form is: $Y_i = \rho_0 + \rho_1 Z_i + e_i$ The IV estimate is: $\beta_{IV} = \frac{\rho_1}{\pi_1}$ This estimates a local average treatment effect for compliers under additional assumptions. Compliers are units whose treatment status is changed by assignment. In a job training experiment, compliers are people who attend training if offered but do not attend if not offered. The IV interpretation requires assumptions such as:

Relevance: assignment affects treatment receipt.
Independence: assignment is random.
Exclusion restriction: assignment affects outcomes only through treatment receipt.
Monotonicity: assignment does not make anyone less likely to receive treatment.

The distinction between assignment and treatment receipt is crucial. If researchers compare actual participants to actual nonparticipants, they may reintroduce selection bias. Even inside an RCT, actual participation may be endogenous if motivated people are more likely to comply. Therefore, when noncompliance exists, assignment-based analysis is usually safer than participation-based analysis.

7.9 Types of noncompliance

Noncompliance can take several forms.

One-sided noncompliance. One-sided noncompliance occurs when control units cannot receive treatment, but some assigned treatment units fail to take it up. Example:
Workers assigned to treatment are offered training.
Some do not attend.
Workers assigned to control have no access to training.

In this case: $D_i=0 \quad \text{for all units with } Z_i=0$ but some units with $Z_i=1$ also have $D_i=0$.

Two-sided noncompliance. Two-sided noncompliance occurs when some assigned treatment units do not receive treatment and some assigned control units do receive treatment. Example:
Some treatment workers do not attend the program.
Some control workers find a similar program elsewhere.

This is common in social programs, education interventions, and health settings.

Partial compliance. Sometimes treatment is not binary. Units may receive different amounts of treatment. Example:
Some students attend all tutoring sessions.
Some attend only half.
Some attend none.

In such cases, researchers must distinguish assignment, eligibility, take-up, attendance, dosage, and exposure. The more complex compliance becomes, the more carefully the estimand must be defined.

7.10 Attrition

Attrition occurs when outcome data are missing for some units after treatment assignment. For example, in a job training RCT, some workers may not respond to the follow-up survey. In an education experiment, some students may leave the school district. In a health experiment, some patients may drop out before final measurement. Attrition threatens internal validity if it is related to treatment status and potential outcomes. Let $R_i=1$ indicate that the outcome is observed, and $R_i=0$ indicate that it is missing. If outcome observation is independent of treatment and potential outcomes, attrition may be less concerning. But if: $R_i \not\perp (Y_i(1),Y_i(0)) \mid D_i$ then attrition can bias estimates. Example: Suppose a job training program performs poorly for some participants, and those participants are less likely to respond to the follow-up survey. Then observed participants may look more successful than the full treatment group. Researchers often assess attrition by asking:

Are attrition rates different between treatment and control groups?
Are attritors different from non-attritors at baseline?
Does treatment predict attrition?
How sensitive are results to assumptions about missing outcomes?

Possible responses include:

intensive tracking,
administrative data linkage,
inverse probability weighting,
bounding exercises,
Lee bounds,
multiple imputation,
sensitivity analysis,
reporting attrition transparently.

Attrition cannot always be solved statistically. Prevention through design is usually better than correction after the fact.

7.11 Spillovers and interference

RCTs often assume no interference between units. That is, one unit’s treatment should not affect another unit’s outcome. In potential outcomes notation, the simple setup assumes: $Y_i = Y_i(D_i)$ But if unit $i$’s outcome depends on other units’ treatment statuses, then potential outcomes should be written as: $Y_i(D_1,D_2,\dots,D_n)$ This creates complications. Examples of spillovers:

A tutoring program may improve classroom peer effects for untreated students.
A vaccination program may protect unvaccinated people through herd immunity.
A job training program may help treated workers get jobs but reduce opportunities for untreated workers.
A policing intervention in one neighborhood may displace crime to nearby neighborhoods.
A cash transfer program may affect local prices and wages.

Spillovers can make the control group an invalid counterfactual because control units are indirectly affected by treatment. There are several design responses:

Cluster randomization. Randomize groups rather than individuals to reduce within-group spillovers. Example: randomize schools instead of students.
Saturation designs. Randomize the share of treated units within groups to estimate spillover effects. Example: treat 25%, 50%, or 75% of households in different villages.
Buffer zones. Separate treated and control units geographically to reduce contamination. Example: exclude households near the border of treated neighborhoods.
Explicit spillover modeling. Estimate both direct and indirect treatment effects. The key lesson is:

If treatment affects untreated units, the meaning of treatment and control changes.

7.12 Hawthorne effects, John Henry effects, and placebo effects

Experiments can change behavior simply because people know they are being studied. A Hawthorne effect occurs when participants change behavior because they are observed or receive attention. Example: workers may become more productive because researchers are monitoring them, not because of the treatment itself. A John Henry effect occurs when control-group members work harder because they know they are in the control group and want to compensate. Example: teachers in control schools may increase effort because they know their school did not receive a new curriculum. A placebo effect occurs when people respond to the belief that they are treated, even if the treatment has no active component. These effects matter because they can change the interpretation of experimental estimates. Possible design responses include:

placebo treatments,
blinded assignment where feasible,
equal attention control groups,
objective outcome measures,
minimizing differential monitoring,
measuring beliefs and expectations.

In social science, full blinding is often impossible. But researchers should still consider whether awareness of assignment affects behavior.

7.13 Internal validity in RCTs

Internal validity asks whether the study credibly estimates a causal effect for the study sample and setting. RCTs often have strong internal validity because randomization addresses confounding. But randomization alone is not enough. Internal validity can still be threatened by:

failed randomization,
noncompliance,
attrition,
spillovers,
measurement error,
differential reporting,
implementation failure,
manipulation of assignment,
multiple hypothesis testing,
selective reporting,
small sample size,
contamination between groups.

A strong RCT report should explain:

how randomization was conducted,
whether assignment was concealed,
whether baseline covariates are balanced,
whether treatment was implemented as planned,
whether compliance was high,
whether attrition was low and balanced,
whether outcomes were measured consistently,
whether spillovers are likely,
whether the analysis follows a pre-specified plan.

Randomization is the beginning of a credible design, not the end of the analysis.

7.14 External validity in RCTs

External validity asks whether the result generalizes beyond the study setting. An RCT may be internally valid but externally limited. For example, suppose a tutoring RCT improves test scores in one urban school district. The estimate may be credible for that setting. But it may not automatically apply to:

rural schools,
different grade levels,
different countries,
different teachers,
larger-scale implementation,
online tutoring,
students with different baseline achievement,
periods with different labor market or family conditions.

External validity depends on:

the population studied,
the treatment version,
the implementing organization,
the institutional setting,
the time period,
the scale of implementation,
the presence of spillovers or equilibrium effects,
the mechanisms through which the treatment works.

A small pilot program may work because it uses unusually skilled staff. When scaled nationally, quality may fall. A job training program may help participants when few people receive it, but if many workers receive the same credential, its labor market value may change. Therefore, RCT evidence should be interpreted as evidence about a particular intervention in a particular context. The right question is not:

Did the RCT work?

The right question is:

What exactly was tested, for whom, under what conditions, and through what mechanisms?

7.15 Individual-level randomization

Individual-level randomization assigns individual units to treatment or control. Examples:

students randomly assigned to tutoring,
workers randomly offered training,
households randomly given information,
patients randomly assigned medication,
consumers randomly shown different prices,
job applicants randomly assigned resume formats.

Individual randomization is often statistically powerful because it can generate many independent treatment-control comparisons. A simple individual-level RCT can be represented as: $Y_i = \alpha + \beta D_i + u_i$ where $D_i$ is randomly assigned. Advantages:

simple design,
high statistical power,
easy interpretation,
direct comparison between treated and control individuals.

Potential problems:

spillovers between treated and control individuals,
resentment or behavior changes among control units,
logistical difficulty if treatment is delivered in groups,
contamination if control units access treatment,
ethical concerns if treatment is visibly withheld.

Individual randomization works best when treatment can be delivered individually and spillovers are limited.

7.16 Cluster randomization

Cluster randomization assigns groups to treatment or control. Examples:

schools randomized to a new curriculum,
villages randomized to a cash transfer program,
clinics randomized to a health protocol,
firms randomized to management consulting,
neighborhoods randomized to policing strategies.

Cluster randomization is useful when treatment operates at the group level or when individual-level randomization would create spillovers. If school-level treatment affects classroom norms, it may be impossible to treat only some students without affecting others. Randomizing schools may be more appropriate. The main statistical issue is that observations within a cluster are correlated. Students in the same school are not independent in the same way that students from different schools are independent. If treatment is assigned at the cluster level, standard errors should usually be clustered at the assignment level. A cluster-level treatment model might be: $Y_{ij} = \alpha + \beta D_j + u_{ij}$ where:

$i$ indexes individuals,
$j$ indexes clusters,
$D_j$ is treatment assignment for cluster $j$.

The effective sample size depends heavily on the number of clusters, not just the number of individuals. A study with 10,000 students but only 10 schools has limited treatment-control variation if schools are the units of assignment. Cluster RCTs therefore require careful power calculations and appropriate inference.

7.17 Encouragement designs

An encouragement design randomly encourages some units to take treatment but does not force treatment. Examples:

randomly sending information about college financial aid,
randomly offering reminders for vaccination,
randomly providing application assistance,
randomly encouraging households to save,
randomly inviting workers to attend training.

Let $Z_i$ be random encouragement, and let $D_i$ be actual treatment take-up. The encouragement may increase the probability of treatment: $\mathbb{E}[D_i \mid Z_i=1] > \mathbb{E}[D_i \mid Z_i=0]$ The ITT estimates the effect of encouragement: $ITT = \mathbb{E}[Y_i \mid Z_i=1] - \mathbb{E}[Y_i \mid Z_i=0]$ If researchers want the effect of actual treatment, they may use encouragement as an instrument for treatment. The IV estimate is: $\frac{\mathbb{E}[Y_i \mid Z_i=1]-\mathbb{E}[Y_i \mid Z_i=0]}{\mathbb{E}[D_i \mid Z_i=1]-\mathbb{E}[D_i \mid Z_i=0]}$ This estimates the effect for compliers, under the IV assumptions. Encouragement designs are especially useful when treatment cannot ethically or practically be forced.

7.18 Phase-in designs

A phase-in design randomly assigns the timing of treatment. Everyone eventually receives treatment, but some units receive it earlier than others. Example: A government plans to roll out an infrastructure program to 100 villages. Because implementation capacity is limited, 50 villages receive the program in year one and 50 receive it in year two. If timing is randomized, year-two villages can serve as controls for year-one villages during the first year. Phase-in designs are ethically attractive when permanently denying treatment is unacceptable. They estimate the effect of earlier access relative to later access. Potential issues:

anticipation effects,
changing implementation quality over time,
spillovers from early to late groups,
difficulty measuring long-run effects if everyone eventually receives treatment,
political pressure affecting rollout order.

The causal interpretation depends on whether timing is truly random and whether late-treatment units provide a credible counterfactual during the comparison period.

7.19 Factorial designs

A factorial design randomizes multiple treatments at the same time. For example, a savings experiment might randomize:

financial education,
text reminders,
matching incentives.

A $2 \times 2$ factorial design with two binary treatments has four groups:

Group	Treatment A	Treatment B
1	0	0
2	1	0
3	0	1
4	1	1

A regression might be: $Y_i = \alpha + \beta_A A_i + \beta_B B_i + \beta_{AB}(A_iB_i) + u_i$ where:

$\beta_A$ is the effect of treatment A when $B_i=0$,
$\beta_B$ is the effect of treatment B when $A_i=0$,
$\beta_{AB}$ captures interaction between the treatments.

Factorial designs are useful because they can test multiple interventions and whether interventions complement or substitute for each other. For example, financial education may have little effect alone, reminders may have little effect alone, but the combination may substantially increase savings. The main challenge is statistical power. Estimating interactions often requires larger samples than estimating main effects.

7.20 Power and minimum detectable effects

Statistical power is the probability that a study detects an effect if the effect truly exists. Power depends on:

sample size,
outcome variability,
treatment assignment ratio,
effect size,
significance level,
clustering,
baseline covariates,
compliance rates,
attrition.

A study with low power may fail to detect meaningful effects. This can lead to a Type 2 error, or false negative. The minimum detectable effect, or MDE, is the smallest effect a study is likely to detect with a given level of power. A simplified expression for the standard error of a difference in means is: $SE(\bar{Y}_1-\bar{Y}_0) = \sqrt{\frac{\sigma_1^2}{n_1}+\frac{\sigma_0^2}{n_0}}$ If the outcome is highly variable, the standard error is larger. If sample size increases, the standard error decreases. Cluster randomization usually increases standard errors because outcomes within clusters are correlated. This is captured by the intraclass correlation coefficient. A rough design effect is: $Design\ Effect = 1 + (m-1)\rho$ where:

$m$ is average cluster size,
$\rho$ is the intraclass correlation.

If $\rho$ is large, adding more individuals within the same cluster may add little independent information. Adding more clusters is often more valuable. Power calculations should be conducted before the experiment, not after results are known.

7.21 Multiple outcomes and multiple testing

RCTs often measure many outcomes. For example, an education experiment may measure:

test scores,
attendance,
graduation,
college enrollment,
behavior,
motivation,
teacher evaluations,
parent involvement.

If researchers test many outcomes, some may appear statistically significant by chance. At a 5% significance level, if 100 unrelated null hypotheses are tested, about 5 may be significant purely by chance on average. This is the multiple testing problem. Researchers can respond by:

pre-specifying primary outcomes,
using outcome indices,
adjusting p-values,
controlling the family-wise error rate,
controlling the false discovery rate,
reporting all outcomes transparently,
distinguishing confirmatory from exploratory analysis.

Pre-analysis plans are often useful. A pre-analysis plan states hypotheses, outcomes, empirical specifications, subgroups, and treatment arms before the data are analyzed. Pre-specification reduces the risk of data mining and selective reporting.

7.22 Heterogeneous treatment effects in RCTs

An RCT estimates an average effect, but treatment effects may differ across units. The individual treatment effect is: $\tau_i = Y_i(1)-Y_i(0)$ The average treatment effect may hide important variation. For example:

tutoring may help low-performing students more than high-performing students,
job training may help younger workers more than older workers,
cash transfers may have larger effects for poorer households,
medical treatments may work differently by baseline risk,
management training may help low-productivity firms more than high-productivity firms.

A conditional average treatment effect is: $CATE(x)=\mathbb{E}[Y_i(1)-Y_i(0) \mid X_i=x]$ Researchers can estimate heterogeneous effects using interactions: $Y_i = \alpha + \beta D_i + \delta X_i + \theta(D_iX_i) + u_i$ Here, $\theta$ captures how the treatment effect varies with $X_i$. However, subgroup analysis must be done carefully. Testing many subgroups can create false discoveries. Subgroup hypotheses are more credible when pre-specified and theoretically motivated. Heterogeneity is important for targeting policy. A program with a modest average effect may be highly valuable for one subgroup and ineffective for another.

7.23 Ethical issues in RCTs

RCTs raise ethical questions because they involve assigning some units to treatment and others to control. Important ethical questions include:

Is it acceptable to withhold treatment from the control group?
Is there genuine uncertainty about whether the treatment works?
Are participants informed and protected?
Are risks reasonable relative to potential benefits?
Are vulnerable populations being exploited?
Is the control condition ethically appropriate?
Will successful interventions eventually be made available more broadly?

The ethical case for an RCT is strongest when there is genuine uncertainty about the treatment’s effects, resources are limited, or a fair allocation mechanism is needed. For example, if a program has limited slots, a lottery may be both ethical and scientifically useful. Phase-in designs can also address ethical concerns by ensuring everyone eventually receives treatment. Ethical evaluation is not separate from research design. A study can be statistically elegant but ethically unacceptable.

7.24 Practical example: job training RCT

Suppose a government wants to evaluate a job training program for unemployed workers. Eligible workers apply to the program, but there are more applicants than available slots. The government randomly assigns applicants to two groups:

treatment group: offered training,
control group: not offered training immediately.

Let $Z_i=1$ if worker $i$ is offered training and $Z_i=0$ otherwise. Let $Y_i$ be annual earnings one year later. The ITT is: $ITT = \mathbb{E}[Y_i \mid Z_i=1] - \mathbb{E}[Y_i \mid Z_i=0]$ Suppose $\mathbb{E}[Y_i \mid Z_i=1] = 34{,}000$ and $\mathbb{E}[Y_i \mid Z_i=0] = 31{,}000$ Then: $ITT = 34{,}000 - 31{,}000 = 3{,}000$ The offer of training increased annual earnings by $3,000 on average. But suppose only 75% of those offered training actually attend, and nobody in the control group attends. Then the first stage is: $\mathbb{E}[D_i \mid Z_i=1] - \mathbb{E}[D_i \mid Z_i=0] = 0.75 - 0 = 0.75$ A simple IV estimate of the effect of attending training for compliers is: $\frac{3{,}000}{0.75}=4{,}000$ This suggests that attending training increased earnings by $4,000 for workers who attended because they were offered the program. However, this interpretation requires assumptions. The offer must affect earnings only through training attendance, and the offer must not discourage anyone from attending. The study should also check:

whether randomization was implemented correctly,
whether baseline characteristics are balanced,
whether attrition differs by assignment,
whether control workers found substitute training,
whether the program displaced nonparticipants in the labor market,
whether effects differ by age, education, or prior earnings,
whether earnings gains exceed program costs.

7.25 Practical example: education RCT

Suppose a school district tests a tutoring program for middle-school students. Students are randomly assigned within schools:

treatment: offered after-school tutoring,
control: business as usual.

Let $Y_i$ be end-of-year math test scores. A simple regression is: $Y_i = \alpha + \beta D_i + u_i$ The coefficient $\beta$ estimates the average effect of being assigned to tutoring. A more precise specification may control for baseline test scores: $Y_i = \alpha + \beta D_i + \gamma Y_{i,baseline} + u_i$ Because baseline scores are measured before treatment, including them can improve precision. But suppose researchers control for attendance during the tutoring period: $Y_i = \alpha + \beta D_i + \delta Attendance_i + u_i$ This may be problematic if tutoring affects attendance. Attendance may be a mediator or post-treatment variable. The study must also consider spillovers. If treated students help untreated classmates, control students may be indirectly affected. If teachers change instruction because some students receive tutoring, the treatment may affect the whole classroom. If spillovers are likely, school-level or classroom-level randomization may be more appropriate than individual-level randomization.

7.26 Practical example: information experiment

Suppose researchers want to know whether providing students with information about financial aid increases college enrollment. They randomly assign high school seniors to receive:

treatment: personalized information about financial aid and application deadlines,
control: no additional information.

The treatment is an information intervention. It does not force students to apply for college or receive aid. The ITT estimates the effect of receiving information: $ITT = \mathbb{E}[Enrollment_i \mid Z_i=1] - \mathbb{E}[Enrollment_i \mid Z_i=0]$ If the information changes applications and applications change enrollment, then the mechanism may be: $Information \rightarrow Applications \rightarrow Aid \rightarrow Enrollment$ Researchers should be careful when controlling for applications or aid receipt. These may be post-treatment mediators. If the goal is the total effect of information, controlling for them may block part of the effect. Information experiments are common because information can be randomized ethically and cheaply. But they estimate the effect of providing information, not necessarily the effect of the underlying program being described.

7.27 How to read an RCT critically.

When reading an RCT, ask the following questions.

What exactly was randomized? Was it treatment receipt, treatment offer, encouragement, eligibility, timing, or information?
What is the unit of randomization? Individuals, classrooms, schools, firms, villages, or markets? The unit of randomization affects both interpretation and standard errors.
What is the treatment? Is the treatment well defined? Was it implemented consistently?
What is the control condition? No treatment, business as usual, placebo, delayed treatment, or another treatment?
What is the estimand? ATE, ITT, TOT, LATE, CATE, or something else?
Was randomization implemented correctly? Was assignment concealed? Were there deviations from the protocol?
Are baseline characteristics balanced? Are treatment and control groups similar before treatment?
Was there noncompliance? Did assigned treatment units receive treatment? Did control units access treatment?
Was there attrition? Are outcome data missing? Is missingness related to treatment or baseline characteristics?
Were there spillovers? Could treated units affect control units?
Were outcomes measured consistently? Could treatment affect reporting, measurement, or observation?
Were multiple outcomes tested? Were primary outcomes pre-specified? Were p-values adjusted?
Is the sample large enough? Was the study powered to detect meaningful effects?
Does the result generalize? To whom, where, and under what implementation conditions?
Is the effect policy-relevant? How large is the effect? What does the program cost? Are benefits larger than costs?

7.28 Common mistakes with RCTs.

Thinking randomization solves every problem. Randomization solves confounding in expectation. It does not automatically solve attrition, noncompliance, spillovers, measurement error, poor implementation, or external validity.
Comparing actual participants to nonparticipants. If take-up is voluntary, actual participation may be endogenous. The randomized comparison is usually based on assignment, not participation.
Ignoring attrition. Missing outcome data can destroy the comparability created by randomization.
Ignoring spillovers. If control units are affected by treated units, the control group may no longer represent the no-treatment counterfactual.
Controlling for post-treatment variables. Post-treatment controls can block causal pathways or introduce bias.
Overgeneralizing from one context. An internally valid RCT estimates an effect for a specific treatment, population, and setting. It does not automatically establish a universal law.
Focusing only on statistical significance. A statistically significant effect may be too small to matter. A statistically insignificant effect may still be economically meaningful if the study is underpowered.
Ignoring costs. A program can have a positive effect but still fail a cost-benefit test.

7.29 Application checklist.

When designing or evaluating an RCT, use the following checklist.

Define the causal question. What intervention is being evaluated? What outcome is affected? For whom?
Define treatment and control. What exactly does the treatment group receive? What exactly does the control group receive?
Define the unit of randomization. Are individuals, groups, institutions, or regions randomized?
Define the estimand. Is the target effect ATE, ITT, TOT, LATE, CATE, or another quantity?
Design the assignment mechanism. How is randomization performed? Is it simple, stratified, blocked, clustered, or phased in?
Check implementation. Was treatment delivered as intended? Did control units remain untreated?
Measure baseline variables. Collect important pre-treatment covariates to assess balance and improve precision.
Measure outcomes consistently. Outcome measurement should not differ systematically by treatment status.
Track compliance. Distinguish assignment, eligibility, take-up, dosage, and exposure.
Track attrition. Report missing data rates and assess whether attrition threatens validity.
Consider spillovers. Could treated units affect control units? Should randomization occur at a higher level?
Plan inference. Use standard errors appropriate to the assignment design, especially with clustering.
Address multiple testing. Pre-specify primary outcomes and adjust for multiple comparisons when appropriate.
Study heterogeneity carefully. Pre-specify subgroup analyses when possible and interpret exploratory heterogeneity cautiously.
Discuss external validity. Explain the population, treatment version, implementation context, and conditions under which results may or may not generalize.
Connect to policy. Compare effect sizes to costs, feasibility, scalability, and distributional consequences.

7.30 Summary

Randomized controlled trials are the benchmark research design for causal inference because random assignment creates comparable treatment and control groups in expectation. With binary treatment, the average treatment effect is: $ATE = \mathbb{E}[Y_i(1)-Y_i(0)]$ Under random assignment: $D_i \perp (Y_i(1),Y_i(0))$ so the causal effect can be identified by the difference in observed average outcomes: $ATE = \mathbb{E}[Y_i \mid D_i=1] - \mathbb{E}[Y_i \mid D_i=0]$ In practice, many RCTs estimate an intent-to-treat effect: $ITT = \mathbb{E}[Y_i \mid Z_i=1] - \mathbb{E}[Y_i \mid Z_i=0]$ where $Z_i$ is assignment or offer of treatment. RCTs can still face serious problems, including noncompliance, attrition, spillovers, measurement error, multiple testing, low power, and limited external validity. Therefore, a good RCT is not merely randomized. It is carefully designed, implemented, analyzed, and interpreted. The central lesson is:

Randomization is powerful because it creates a credible counterfactual by design. But credible experimental evidence still requires clear treatment definition, careful implementation, valid measurement, appropriate inference, and honest interpretation of scope and limitations.

8. Natural Experiments and Quasi-Experimental Design

8.1 Why natural experiments matter

Randomized controlled trials are the conceptual benchmark for causal inference because treatment assignment is controlled by the researcher. When treatment is randomly assigned, the treated and control groups are comparable in expectation, so the control group can provide a credible estimate of the treated group’s missing counterfactual. But many of the most important questions in economics cannot be answered using researcher-controlled randomization. Researchers usually cannot randomly assign:

years of schooling,
unemployment,
immigration,
incarceration,
exposure to pollution,
exposure to war,
neighborhood of childhood,
minimum wage laws,
tax rates,
monetary policy,
trade exposure,
family structure,
health shocks,
policing intensity,
access to infrastructure.

Some interventions are unethical to randomize. Others are too expensive, too large, too political, or too slow-moving. In many cases, the policy has already occurred, and the researcher must study it after the fact. This is where natural experiments and quasi-experimental designs become central. A natural experiment occurs when the world creates variation in treatment that is plausibly similar to random assignment. The researcher does not control the assignment process, but some rule, event, shock, policy, institution, or accident generates variation that can be used for causal inference. The purpose of a natural experiment is to approximate the logic of an experiment using observational data. The key question is always:

What is the source of variation in treatment, and why is that variation plausibly exogenous?

A natural experiment is not defined by whether the setting feels unusual or dramatic. It is defined by whether the treatment variation helps construct a credible counterfactual.

8.2 Natural experiments versus ordinary observational studies

An ordinary observational study uses data in which treatment was not assigned by the researcher. For example, a researcher might compare people who attended college with people who did not, or cities with high police levels to cities with low police levels. The difficulty is that treatment status may be related to potential outcomes. People who attend college differ from people who do not. Cities with more police differ from cities with fewer police. Countries that receive aid differ from countries that do not. A natural experiment is a special kind of observational setting in which treatment variation arises from a source that is plausibly unrelated to the unobserved determinants of the outcome. For example:

A draft lottery assigns some people a higher probability of military service.
A school-entry cutoff makes some children eligible to start school earlier than others.
A judge-assignment system exposes similar defendants to judges with different sentencing tendencies.
A policy rollout affects some regions earlier than others.
A weather shock affects agricultural income in some areas but not others.
An administrative eligibility threshold gives benefits to people just below or above a cutoff.

The difference between a weak observational study and a strong natural experiment is not the dataset. It is the credibility of the assignment process. A weak design says:

We compare treated and untreated units and control for observable differences.

A stronger natural-experiment design says:

Treatment changed because of a specific rule, shock, or institutional process that is plausibly unrelated to the potential outcomes except through treatment.

That design-based argument is the heart of credible empirical economics.

8.3 The experimental ideal and the quasi-experimental substitute

To understand quasi-experimental design, begin with the ideal experiment. Suppose we want to estimate the average treatment effect: $ATE = \mathbb{E}[Y_i(1)-Y_i(0)].$ In a randomized experiment, treatment assignment satisfies: $D_i \perp (Y_i(1),Y_i(0)).$ This means treatment assignment is independent of potential outcomes. Under random assignment: $\mathbb{E}[Y_i(1) \mid D_i=1] = \mathbb{E}[Y_i(1) \mid D_i=0]$ and: $\mathbb{E}[Y_i(0) \mid D_i=1] = \mathbb{E}[Y_i(0) \mid D_i=0].$ Therefore, the observed treated-control difference identifies the causal effect: $\mathbb{E}[Y_i \mid D_i=1]-\mathbb{E}[Y_i \mid D_i=0] = ATE.$ In a quasi-experiment, treatment is not randomly assigned by the researcher. Instead, the researcher argues that some part of treatment variation behaves as if it were random, conditional on the design. The ideal randomized experiment asks:

What would happen if the researcher randomly assigned treatment?

The quasi-experimental design asks:

Did some real-world process assign treatment in a way that approximates random assignment for the causal question at hand?

This is why quasi-experimental research is sometimes called design-based causal inference. The credibility of the result comes less from the complexity of the statistical model and more from the source of variation used to identify the effect.

8.4 What makes a natural experiment credible?

A credible natural experiment usually has several features.

A clearly defined treatment. The treatment must be specific enough to support a causal interpretation. Instead of asking:

What is the effect of education?

we might ask:

What is the effect of being required by a compulsory schooling law to remain in school until age sixteen rather than age fourteen?

Instead of asking:

What is the effect of healthcare access?

we might ask:

What is the effect of gaining Medicaid eligibility at a specific income threshold?

Instead of asking:

What is the effect of policing?

we might ask:

What is the effect of an increase in patrol hours caused by an administrative redeployment rule?

Causal inference requires a treatment that can be connected to a real or hypothetical intervention.

A credible comparison group. The comparison group must approximate the missing counterfactual. If a state raises its minimum wage, the untreated comparison states should represent what would have happened to the treated state in the absence of the policy. If students just below a scholarship cutoff receive aid, students just above the cutoff should represent what would have happened to treated students without aid. The comparison does not need to be perfect in every respect. It must be credible for the specific counterfactual claim required by the design.
A source of plausibly exogenous variation. The treatment variation should come from a source that is not chosen in response to the potential outcomes. Examples include:
random lotteries,
administrative cutoffs,
unexpected policy changes,
weather shocks,
institutional assignment rules,
staggered policy rollouts,
eligibility thresholds,
geographic borders,
judge or examiner assignment,
cohort-based rules.

The word “exogenous” does not mean magical or automatically valid. It means external to the unobserved determinants of the outcome in the model being used. A researcher must explain why the variation is plausibly unrelated to the error term, potential outcomes, or untreated counterfactual trend.

A transparent identifying assumption. Every natural experiment depends on an assumption. Examples:
Difference-in-differences requires parallel trends.
Regression discontinuity requires continuity of potential outcomes at the cutoff.
Instrumental variables require relevance and exclusion.
Synthetic control requires that a weighted combination of control units approximates the treated unit’s untreated counterfactual.
Matching requires no unobserved confounding conditional on observed covariates.

The assumption should be stated explicitly. A study is more credible when the reader can clearly see what must be true for the estimate to be causal.

Evidence supporting the assumption. The identifying assumption is often not directly testable. However, researchers can provide indirect evidence. Examples:
balance tests,
pre-trend tests,
placebo outcomes,
placebo treatment dates,
density tests around cutoffs,
institutional detail,
robustness to alternative specifications,
sensitivity analysis,
falsification tests,
checks for manipulation,
checks for spillovers.

None of these tests proves the design is valid. But together they can make the identifying assumption more or less plausible.

8.5 As-good-as-random variation

The phrase “as-good-as-random” is central to natural experiments. It does not mean treatment was literally randomized. It means that, for the purpose of estimating a particular causal effect, the relevant variation in treatment is plausibly unrelated to potential outcomes. Formally, in a simple binary-treatment setting, random assignment gives: $D_i \perp (Y_i(1),Y_i(0)).$ A natural experiment tries to justify something similar, sometimes unconditionally and sometimes after conditioning on design features: $D_i \perp (Y_i(1),Y_i(0)) \mid X_i.$ In words, treatment is independent of potential outcomes after accounting for relevant covariates or institutional rules. But many quasi-experimental methods use more specialized assumptions. In regression discontinuity, treatment may not be independent of potential outcomes overall. Students below a tutoring cutoff may differ from students far above the cutoff. The claim is only local:

Units just below and just above the cutoff are comparable.

In difference-in-differences, treated and control groups may differ in levels. The claim is about trends:

In the absence of treatment, treated and control groups would have followed parallel trends.

In instrumental variables, treatment may be highly endogenous. The claim is about the instrument:

The instrument shifts treatment but is otherwise unrelated to potential outcomes.

Thus, “as-good-as-random” must always be interpreted relative to a particular design.

8.6 Common sources of natural-experimental variation

Natural experiments arise from many sources. The most important thing is not the label attached to the source, but whether the source creates credible treatment variation.

Lotteries. Lotteries are among the cleanest natural experiments because they resemble explicit random assignment. Examples:
military draft lotteries,
school admission lotteries,
housing voucher lotteries,
visa lotteries,
oversubscribed program lotteries,
lottery-based allocation of public resources.

If lottery assignment is truly random and affects treatment, it can be used to estimate causal effects. However, even lottery designs require care. A lottery may assign eligibility or an offer, not actual treatment. Some people who win may not take up the treatment, and some who lose may obtain treatment elsewhere. In that case, the lottery may identify an intent-to-treat effect or serve as an instrument for actual treatment. Let $Z_i$ denote winning a lottery and $D_i$ denote actual treatment. The effect of winning the lottery on the outcome is: $ITT = \mathbb{E}[Y_i \mid Z_i=1]-\mathbb{E}[Y_i \mid Z_i=0].$ If winning the lottery affects actual treatment, $Z_i$ may be used as an instrument for $D_i$.

Administrative cutoffs. Many policies use thresholds or cutoffs. Examples:
income thresholds for benefits,
age cutoffs for school entry,
test-score cutoffs for scholarships,
population thresholds for regulations,
poverty thresholds for program eligibility,
credit-score thresholds for loan approval.

If treatment changes discontinuously at a cutoff, researchers may use regression discontinuity. The key idea is that units just below and just above the cutoff may be similar except for treatment eligibility. For a cutoff $c$ and running variable $R_i$, treatment may be assigned as: $D_i = 1\{R_i \geq c\}.$ The RD design estimates the jump in outcomes at the cutoff: $\tau_{RD} = \lim_{r \downarrow c} \mathbb{E}[Y_i \mid R_i=r] - \lim_{r \uparrow c} \mathbb{E}[Y_i \mid R_i=r].$ This design is credible when potential outcomes would have evolved smoothly through the cutoff in the absence of treatment.

Staggered policy rollouts. Policies are often introduced in different places at different times. Examples:
states adopt minimum wage laws in different years,
countries expand health insurance at different times,
cities introduce policing reforms sequentially,
school districts adopt curricula in different years,
regions receive infrastructure programs in phases.

Staggered rollout can support difference-in-differences designs if untreated or not-yet-treated units provide a credible counterfactual for treated units. The key identifying assumption is usually a version of parallel trends: $\mathbb{E}[Y_{it}(0)-Y_{i,t-1}(0) \mid D_i=1] = \mathbb{E}[Y_{it}(0)-Y_{i,t-1}(0) \mid D_i=0].$ In words, treated and control units would have had the same average untreated change in outcomes. Staggered adoption requires special care when treatment effects vary over time or across cohorts. Traditional two-way fixed effects estimators can mix comparisons in problematic ways. Modern difference-in-differences methods often compare treated units to not-yet-treated or never-treated units and estimate cohort-time-specific effects.

Geographic borders. Policy rules often change at borders. Examples:
minimum wages differ across state borders,
tax rates differ across jurisdictions,
school policies differ across district boundaries,
environmental regulations differ across counties,
healthcare access differs across administrative boundaries.

Border designs compare units close to opposite sides of a boundary. The idea is that nearby units may share similar economic, social, geographic, and demographic conditions, while facing different policies. The credibility of a border design depends on whether units near the border are comparable and whether other policies also change at the border. A simple border comparison can be written as: $Y_i = \alpha + \tau D_i + f(Location_i) + u_i,$ where $D_i$ indicates the policy side of the border and $f(Location_i)$ controls flexibly for geography. Potential threats include sorting, multiple policy differences at the same boundary, spillovers across the border, and local economic integration.

Weather and environmental shocks. Weather is often used as a source of plausibly exogenous variation. Examples:
rainfall shocks and agricultural output,
temperature shocks and labor productivity,
drought and migration,
hurricanes and local economic activity,
heat exposure and health outcomes,
snowstorms and transportation disruptions.

The appeal is that weather is typically not chosen by individuals, firms, or governments. However, weather is not automatically exogenous for every question. For example, richer households may live in areas less exposed to flooding, farmers may adapt through irrigation, and governments may invest in protective infrastructure in high-risk areas. Long-run climate exposure may be correlated with settlement patterns, income, institutions, or adaptation. A weather shock is most credible when it represents unexpected short-run variation around local norms. Researchers often estimate models like: $Y_{it} = \alpha_i + \lambda_t + \beta WeatherShock_{it} + u_{it},$ where $\alpha_i$ are unit fixed effects and $\lambda_t$ are time fixed effects. The unit fixed effects compare each place to itself, and time fixed effects absorb shocks common to all places in a period.

Institutional assignment rules. Some institutions assign people to decision-makers or treatment environments in ways that appear quasi-random. Examples:
judges assigned to cases,
examiners assigned to disability claims,
caseworkers assigned to welfare recipients,
teachers assigned to classrooms,
doctors assigned to patients,
loan officers assigned to applicants.

If assignment is plausibly random or conditionally random, variation in decision-maker tendencies can be used for causal inference. For example, some judges may be more likely to impose incarceration than others. If defendants are quasi-randomly assigned to judges, judge harshness can be used as an instrument for incarceration. Let $Z_i$ be the tendency of assigned judge $j$ to impose incarceration. Let $D_i$ be whether defendant $i$ is incarcerated. Let $Y_i$ be a later outcome, such as employment or reoffending. A judge IV design requires:

judge assignment affects incarceration,
judge assignment is as-good-as-random conditional on court and timing,
judge assignment affects later outcomes only through incarceration,
monotonicity or an equivalent condition for LATE interpretation.

These designs can be powerful, but they rely heavily on institutional detail.

8.7 Quasi-experimental designs

A quasi-experimental design is a research design that approximates experimental logic without researcher-controlled random assignment. Common quasi-experimental methods include:

difference-in-differences,
regression discontinuity,
instrumental variables,
synthetic control,
event studies,
interrupted time series,
panel fixed effects,
matching and weighting,
border discontinuity designs,
judge or examiner designs.

These methods differ in how they construct the missing counterfactual.

Design	Counterfactual logic
Difference-in-differences	Control group trend approximates treated group’s untreated trend
Regression discontinuity	Units just across the cutoff approximate treated units’ counterfactual
Instrumental variables	Instrument-induced treatment variation isolates causal effect
Synthetic control	Weighted combination of untreated units approximates treated unit’s counterfactual
Event study	Dynamic comparisons before and after treatment assess timing and pre-trends
Interrupted time series	Pre-treatment trend projects untreated counterfactual after intervention
Fixed effects	Units are compared to themselves over time
Matching/weighting	Treated and untreated units with similar observed covariates are compared

No design is automatically valid. Each design is valid only if its identifying assumptions are credible.

8.8 Difference-in-differences as a quasi-experiment

Difference-in-differences, or DiD, is used when some units are treated and others are not, and outcomes are observed before and after treatment. The simplest two-group, two-period setup has:

treated group $T$,
control group $C$,
pre-treatment period $0$,
post-treatment period $1$.

The DiD estimand is: $\tau_{DiD} = \left(\mathbb{E}[Y_{T1}]-\mathbb{E}[Y_{T0}]\right) - \left(\mathbb{E}[Y_{C1}]-\mathbb{E}[Y_{C0}]\right).$ This subtracts the control group’s change from the treated group’s change. The central assumption is parallel trends: $\mathbb{E}[Y_{T1}(0)-Y_{T0}(0)] = \mathbb{E}[Y_{C1}(0)-Y_{C0}(0)].$ In words, absent treatment, the treated group would have changed like the control group. DiD is useful because treated and control groups do not need to have equal outcome levels before treatment. They only need to have comparable trends in the absence of treatment. However, DiD can fail if:

treated and control groups had different pre-existing trends,
another shock affected only the treated group at the treatment time,
treatment timing was chosen because of expected outcome changes,
there were spillovers from treated to control units,
group composition changed over time,
people anticipated the policy,
treatment effects varied in ways that make simple estimators misleading.

A good DiD study usually includes graphical pre-trend analysis, event-study estimates, placebo tests, alternative control groups, and institutional justification for the timing of treatment.

8.9 Regression discontinuity as a quasi-experiment

Regression discontinuity, or RD, uses a cutoff rule that changes treatment assignment. Suppose students with test scores below 60 receive tutoring: $D_i = 1\{Score_i < 60\}.$ Students with scores of 59 and 61 may be very similar. If tutoring changes discontinuously at 60, then comparing students near the cutoff can estimate the local effect of tutoring. The RD estimand is: $\tau_{RD} = \lim_{x \uparrow c}\mathbb{E}[Y_i \mid X_i=x] - \lim_{x \downarrow c}\mathbb{E}[Y_i \mid X_i=x],$ with the sign depending on which side receives treatment. The key assumption is continuity of potential outcomes: $\lim_{x \uparrow c}\mathbb{E}[Y_i(0) \mid X_i=x] = \lim_{x \downarrow c}\mathbb{E}[Y_i(0) \mid X_i=x]$ and similarly for $Y_i(1)$ where relevant. In words, absent treatment, outcomes would have changed smoothly through the cutoff. RD can be very credible near the threshold, but it estimates a local effect. A scholarship RD at a GPA cutoff of 3.5 estimates the effect for students near 3.5, not necessarily for students with GPAs of 2.0 or 4.0. Threats to RD include:

manipulation of the running variable,
sorting around the cutoff,
other policies changing at the same cutoff,
inappropriate bandwidth choice,
misspecified functional form,
low statistical power near the cutoff.

Common diagnostics include density tests, covariate balance near the cutoff, placebo cutoffs, alternative bandwidths, and graphical analysis.

8.10 Instrumental variables as a quasi-experiment

Instrumental variables, or IV, are used when treatment is endogenous but some variable shifts treatment in a plausibly exogenous way. Suppose the causal model is: $Y_i = \alpha + \beta D_i + u_i,$ but treatment is endogenous: $\operatorname{Cov}(D_i,u_i) \neq 0.$ An instrument $Z_i$ can help if it satisfies two core conditions. First, relevance: $\operatorname{Cov}(Z_i,D_i) \neq 0.$ The instrument must affect treatment. Second, exclusion: $Z_i \rightarrow D_i \rightarrow Y_i,$ with no direct path from $Z_i$ to $Y_i$ except through $D_i$. Examples of instruments include:

draft lottery number for military service,
distance to college for college attendance,
judge assignment for incarceration,
program eligibility for participation,
rainfall shocks for agricultural income,
policy eligibility rules for treatment take-up.

IV identifies the effect for compliers under additional assumptions. Compliers are units whose treatment status changes because of the instrument. The IV estimand in the simple binary case is: $\frac{\mathbb{E}[Y_i \mid Z_i=1]-\mathbb{E}[Y_i \mid Z_i=0]} {\mathbb{E}[D_i \mid Z_i=1]-\mathbb{E}[D_i \mid Z_i=0]}.$ This is the reduced-form effect of the instrument on the outcome divided by the first-stage effect of the instrument on treatment. IV can be powerful, but exclusion restrictions are often controversial. A variable that affects treatment may also affect the outcome through other channels. The question is not only whether the instrument predicts treatment. The deeper question is whether the instrument affects the outcome only through the treatment.

8.11 Synthetic control

Synthetic control is often used when a policy affects one unit or a small number of units, and there are many possible control units. For example, suppose one state adopts a major policy reform. We want to know what would have happened to that state without the reform. Synthetic control constructs a weighted average of untreated units to approximate the treated unit before treatment. Let unit $1$ be treated, and units $2,\dots,J+1$ be untreated. A synthetic control uses weights $w_j$ such that: $w_j \geq 0$ and $\sum_{j=2}^{J+1} w_j = 1.$ The synthetic untreated outcome for treated unit $1$ after treatment is: $\sum_{j=2}^{J+1} w_j Y_{jt}.$ The estimated treatment effect at time $t$ is: $\hat{\tau}_{1t} = Y_{1t} - \sum_{j=2}^{J+1} w_j Y_{jt}.$ The credibility of synthetic control depends on whether the weighted control group closely matches the treated unit before treatment and whether no unobserved post-treatment shocks differentially affect the treated unit. Advantages of synthetic control include transparency, visual diagnostics, and usefulness for case studies. Threats include poor pre-treatment fit, spillovers, too few control units, unobserved shocks after treatment, and researcher discretion in choosing predictors or donor pools.

8.12 Event studies

An event study estimates how outcomes evolve before and after treatment. Event studies are often used with policy changes or staggered adoption settings. A typical event-study model includes leads and lags of treatment: $Y_{it} = \alpha_i + \lambda_t + \sum_{k \neq -1}\beta_k \mathbf{1}\{t-T_i=k\} + u_{it},$ where:

$\alpha_i$ are unit fixed effects,
$\lambda_t$ are time fixed effects,
$T_i$ is the treatment time for unit $i$,
$k$ indexes time relative to treatment,
$\beta_k$ traces outcomes before and after treatment.

Pre-treatment coefficients help assess whether treated units were already trending differently before treatment. If the coefficients for periods before treatment are far from zero, the design may violate the parallel trends assumption. Post-treatment coefficients show whether effects appear, grow, fade, or persist. Event studies are useful because they make timing visible. But they do not automatically solve identification problems. If treatment timing is endogenous or if comparison groups are inappropriate, the event-study estimates may still be biased. In staggered adoption settings, event studies must be implemented carefully because already-treated units may serve as controls for later-treated units in problematic ways under traditional two-way fixed effects models.

8.13 Interrupted time series

Interrupted time series designs study outcomes before and after a policy or intervention in a single unit or group. For example, suppose a city implements a new traffic safety law in 2020. A researcher may examine traffic fatalities before and after 2020. A simple interrupted time series model might be: $Y_t = \alpha + \beta t + \tau Post_t + u_t,$ where $Post_t$ equals one after the intervention. A more flexible model may allow both a level change and a slope change: $Y_t = \alpha + \beta t + \tau Post_t + \delta(t \times Post_t) + u_t.$ Here:

$\tau$ captures an immediate level shift,
$\delta$ captures a change in trend after the intervention.

The key assumption is that the pre-treatment trend provides a credible forecast of the untreated post-treatment counterfactual. Interrupted time series designs are vulnerable to other events occurring at the same time, changes in measurement, seasonality, autocorrelation, and nonlinear trends. They are stronger when:

there are many pre-treatment periods,
the intervention occurs at a clearly defined time,
no other major changes occur simultaneously,
outcomes are measured consistently,
robustness checks use alternative trends and placebo dates,
comparison units are added when possible.

8.14 Matching and weighting as quasi-experimental tools

Matching and weighting try to make treated and untreated groups comparable on observed characteristics. The key assumption is unconfoundedness: $(Y_i(1),Y_i(0)) \perp D_i \mid X_i.$ This means treatment is independent of potential outcomes after conditioning on covariates $X_i$. Under this assumption, treated and untreated units with the same covariates can be compared causally. Matching pairs treated units with similar untreated units. Weighting assigns weights so that the distribution of covariates in the control group resembles the treated group. A common tool is the propensity score: $e(X_i)=P(D_i=1 \mid X_i).$ If treatment is unconfounded conditional on $X_i$, then it is also unconfounded conditional on the propensity score under standard conditions: $(Y_i(1),Y_i(0)) \perp D_i \mid e(X_i).$ Matching and weighting can improve balance on observables, but they do not solve unobserved confounding. If treated and untreated units differ in motivation, ability, private information, institutional quality, or other unmeasured factors, matching may still be biased. These methods are most credible when the researcher has rich pre-treatment covariates and a strong argument that selection occurs on observables.

8.15 Panel fixed effects as a quasi-experimental tool

Panel data observe the same units over time. Fixed effects compare each unit to itself. A simple fixed-effects model is: $Y_{it} = \alpha_i + \lambda_t + \beta D_{it} + u_{it},$ where:

$\alpha_i$ are unit fixed effects,
$\lambda_t$ are time fixed effects,
$D_{it}$ is treatment,
$\beta$ is the estimated treatment effect.

Unit fixed effects remove time-invariant differences across units. Time fixed effects remove shocks common to all units in a given period. Fixed effects are useful when unobserved confounders are stable over time. For example, if high-ability workers are more likely to get training and ability is constant, worker fixed effects can remove that source of bias by comparing workers to themselves before and after training. However, fixed effects do not remove time-varying confounders. If workers enter training after a negative earnings shock, or if firms adopt technology when demand is rising, then fixed effects alone may still be biased. The identifying assumption is that, after fixed effects and controls, treatment timing is unrelated to unobserved time-varying determinants of outcomes. This is often a strong assumption.

8.16 The role of institutional knowledge

Natural experiments depend heavily on institutional knowledge. A statistical pattern alone rarely proves that variation is exogenous. The researcher must understand how treatment was assigned. Important institutional questions include:

Who created the rule or policy?
Why was it created?
When was it implemented?
Who was eligible?
Could individuals manipulate eligibility?
Was treatment actually received by eligible units?
Were there exceptions?
Did other policies change at the same time?
Did affected units anticipate the change?
Were there spillovers to untreated units?
Was enforcement uniform?
Did measurement change after the policy?

For example, an age cutoff may seem arbitrary. But if parents can delay school entry strategically, the cutoff may not create as-good-as-random variation. A policy rollout may seem useful, but if high-need areas received the policy first, rollout timing may be endogenous. Judge assignment may seem random, but if certain case types are assigned to particular judges, the design may be biased. Institutional detail is not decorative background. It is part of identification.

8.17 Diagnostics and falsification tests

Because identifying assumptions are often not directly testable, researchers use diagnostics and falsification tests to probe credibility.

Balance tests. Balance tests check whether treated and control groups look similar on pre-treatment covariates. In an RCT, balance is expected because of randomization. In a natural experiment, balance can support the claim that treatment variation is as-good-as-random. A balance test might estimate: $X_i = \alpha + \pi D_i + v_i,$ where $X_i$ is a pre-treatment covariate. If $\pi$ is large, treated and control units differ before treatment. Balance tests are not definitive. Groups can be balanced on observed covariates but unbalanced on unobserved covariates. Still, imbalance on observed pre-treatment variables is a warning sign.
Pre-trend tests. Pre-trend tests are especially important in DiD and event-study designs. They ask whether treated and control units were already evolving differently before treatment. If outcomes diverged before treatment, the control group may not represent the treated group’s untreated counterfactual. However, failure to reject pre-trend differences does not prove parallel trends. Pre-trend tests may have low power. Graphical evidence and institutional reasoning remain important.
Placebo outcomes. A placebo outcome is an outcome that should not be affected by the treatment. For example, if a job training program supposedly affects earnings, it should not affect workers’ pre-treatment earnings. If it appears to affect pre-treatment earnings, that suggests treated and control units differed before treatment.
Placebo treatment dates. A placebo treatment date assigns treatment to a time before the actual intervention. If the estimated effect appears before treatment occurred, the design may be capturing trends or shocks rather than treatment.
Manipulation tests. In RD designs, manipulation tests check whether units sort around the cutoff. If many units appear just on the favorable side of the threshold, the running variable may have been manipulated.
Sensitivity analysis. Sensitivity analysis asks how strong violations of assumptions would need to be to change the conclusion. For example:
How large would unobserved confounding need to be?
How sensitive are results to alternative control groups?
How sensitive are results to bandwidth choice?
How sensitive are results to functional form?
How sensitive are results to excluding certain units or periods?

A result is more credible when it is not driven by one fragile specification.

8.18 Common threats to quasi-experimental validity

Quasi-experimental designs can fail in many ways.

Endogenous treatment timing. Treatment timing is endogenous when units receive treatment because of expected or ongoing changes in outcomes. For example, a city may adopt a policing reform because crime is already rising. A state may raise the minimum wage during a strong labor market. A school may adopt tutoring because test scores are falling. If treatment timing responds to outcome trends, before-after or DiD comparisons may be biased.
Differential trends. Treated and control units may have different underlying trends even without treatment. For example, urban and rural areas may have different employment trends. States adopting certain policies may already be on different trajectories. Firms adopting technology may already be growing faster. Differential trends are a major threat to DiD and event-study designs.
Anticipation effects. Units may change behavior before treatment begins if they anticipate it. Examples:
firms adjust hiring before a minimum wage increase,
households buy goods before a tax increase,
students change effort before a scholarship cutoff,
investors respond before a regulation takes effect.

Anticipation makes it difficult to define the pre-treatment period.

Spillovers. Treatment may affect untreated units. Examples:
job training participants may compete with nonparticipants for jobs,
policing in one neighborhood may shift crime elsewhere,
vaccination protects unvaccinated people,
school reforms affect peer networks,
regional policies affect neighboring labor markets.

Spillovers violate the assumption that the control group represents the untreated counterfactual.

Sorting and manipulation. People may sort into treatment status. Examples:
families move across school district boundaries,
firms adjust size to avoid regulation,
students retake exams to cross scholarship cutoffs,
households alter reported income to qualify for benefits.

Sorting undermines designs based on cutoffs, borders, or eligibility rules.

Multiple simultaneous treatments. If several policies change at once, it may be difficult to isolate the effect of one policy. For example, a state may raise the minimum wage while also expanding tax credits, changing enforcement, and experiencing a labor market boom. The estimated effect may combine several changes.
Measurement changes. A policy may change how outcomes are measured. For example, policing reforms may change crime reporting. School accountability policies may change test-taking behavior. Healthcare expansions may increase diagnosis rates without changing underlying health. Measurement changes can be mistaken for real outcome changes.
External validity limitations. Many quasi-experimental estimates are local. An RD estimate applies near the cutoff. An IV estimate applies to compliers. A policy-shock estimate applies to the affected setting and time period. A synthetic control estimate applies to one treated unit. This does not make the estimates useless. It means they must be interpreted carefully.

8.19 Identification, estimation, and inference in quasi-experiments

It is useful to distinguish three tasks.

Identification. Identification asks whether the causal effect can be recovered from observed data under assumptions. For example:
DiD identifies the effect if parallel trends holds.
RD identifies the local effect if potential outcomes are continuous at the cutoff.
IV identifies LATE if relevance, independence, exclusion, and monotonicity hold.

Identification is about the logic of the design.

Estimation. Estimation is the statistical procedure used to calculate the effect from data. Examples:
OLS with fixed effects,
local linear regression,
two-stage least squares,
weighted least squares,
matching estimators,
synthetic control weights.

A sophisticated estimator cannot rescue a weak identification strategy. The estimator implements the design; it does not create exogeneity by itself.

Inference. Inference asks how uncertain the estimate is. Important issues include:
clustering standard errors,
serial correlation,
small numbers of treated units,
randomization inference,
bootstrap methods,
multiple testing,
spatial correlation.

A quasi-experimental estimate can be biased even if standard errors are tiny. Conversely, a credible design may produce wide confidence intervals if the data are limited. Credible empirical work requires all three: identification, estimation, and inference.

8.20 Practical example: compulsory schooling laws

Suppose we want to estimate the effect of education on earnings. A naive regression is: $Wage_i = \alpha + \beta Education_i + u_i.$ This regression is likely biased because education is endogenous. People with more schooling may differ in ability, family background, motivation, location, health, and networks. A natural experiment might use compulsory schooling laws. Suppose a reform required some cohorts to stay in school longer than earlier cohorts. The policy creates variation in education that is not purely chosen by individuals. A researcher might compare cohorts affected by the law to cohorts not affected by the law, possibly across regions with different timing of reforms. The identifying argument is:

The schooling reform changed education for some cohorts for reasons unrelated to their individual potential earnings, except through schooling.

This can support IV or DiD designs. However, the design must address threats:

Did the reform affect other things besides schooling?
Were affected cohorts different for other reasons?
Did labor markets change at the same time?
Did enforcement vary across regions?
Did the reform affect only certain types of students?

If used as an instrument, the reform may estimate a LATE for students whose schooling was changed by the law. It may not estimate the return to education for all students. A careful conclusion would say:

Under the assumption that the compulsory schooling reform affected earnings only through schooling and was otherwise unrelated to cohort-specific earnings trends, the estimate captures the return to additional schooling for individuals whose education increased because of the reform.

That is more precise than simply saying:

Education raises earnings by $\beta$.

8.21 Practical example: minimum wage changes

Suppose one state raises the minimum wage while neighboring states do not. We want to estimate the effect on employment. A simple before-after comparison in the treated state is not enough: $\mathbb{E}[Employment_{after}]-\mathbb{E}[Employment_{before}].$ Employment may have changed because of the business cycle, seasonality, industry shifts, migration, or other policies. A difference-in-differences design compares the treated state’s change to the control states’ change: $\tau_{DiD} = (Employment_{treated,after}-Employment_{treated,before}) - (Employment_{control,after}-Employment_{control,before}).$ The identifying assumption is:

In the absence of the minimum wage increase, employment in the treated state would have followed the same trend as employment in the control states.

Evidence supporting this might include:

similar pre-treatment employment trends,
similar industry composition,
no simultaneous policy changes,
robustness to alternative control states,
event-study coefficients near zero before treatment,
no anticipation effects.

Threats include:

the treated state adopted the policy because its labor market was already changing,
neighboring states were affected by spillovers,
firms adjusted before the law took effect,
industry composition changed differently across states,
other policies changed at the same time.

A strong empirical paper would not merely report a DiD coefficient. It would defend the counterfactual trend assumption.

8.22 Practical example: judge assignment and incarceration

Suppose we want to estimate the effect of incarceration on future employment. A naive comparison between incarcerated and non-incarcerated defendants is likely biased. Defendants who are incarcerated may differ in offense severity, criminal history, employment prospects, legal representation, and other unobserved factors. Some court systems assign cases to judges in a way that is plausibly random within court and time period. Judges may differ in sentencing harshness. This creates a natural experiment. Let:

$Z_i$ be the harshness of the assigned judge,
$D_i$ be whether defendant $i$ is incarcerated,
$Y_i$ be later employment.

Judge harshness can serve as an instrument if:

harsher judges are more likely to incarcerate defendants,
judge assignment is as-good-as-random conditional on court and time,
judge harshness affects later employment only through incarceration,
there are no problematic defiers.

The first stage is: $D_i = \pi_0 + \pi_1 Z_i + X_i'\pi + v_i.$ The second stage is: $Y_i = \alpha + \beta \hat{D}_i + X_i'\gamma + u_i.$ The estimate $\beta$ is interpreted as the effect of incarceration for defendants whose incarceration status was changed by being assigned to a harsher judge. Threats include:

nonrandom case assignment,
judges affecting outcomes through channels other than incarceration,
plea bargaining responses,
differences in case processing,
effects on sentence length rather than incarceration alone.

This example illustrates that natural experiments can be powerful but require detailed knowledge of institutions.

8.23 Practical example: weather shocks and agricultural income

Suppose we want to estimate the effect of agricultural income on schooling decisions. A naive regression of schooling on income may be biased because income is related to parental education, land quality, local institutions, wealth, and preferences. Weather shocks can create plausibly exogenous variation in agricultural income. A researcher might estimate: $Schooling_{it} = \alpha_i + \lambda_t + \beta RainfallShock_{it} + u_{it}.$ If rainfall shocks affect schooling through agricultural income, then rainfall may be used as a reduced-form shock or as an instrument for income. The identifying argument is:

Conditional on location fixed effects and time fixed effects, deviations in rainfall from normal levels are unrelated to other determinants of schooling.

Threats include:

rainfall directly affects school attendance through road conditions or disease,
households adapt through irrigation,
rainfall affects local prices or labor demand beyond household income,
migration responds to weather,
measurement error in rainfall exposure,
long-run climate differences confound cross-sectional comparisons.

Weather is not automatically a valid instrument. The exclusion restriction depends on the outcome and context.

8.24 How to read a quasi-experimental paper.

When reading an empirical paper that claims to use a natural experiment, ask the following questions.

What is the causal question? What treatment effect is the paper trying to estimate? Is the treatment clearly defined? Is the outcome clearly defined?
What is the source of treatment variation? Where does the variation come from? Examples:

lottery,
cutoff,
policy timing,
judge assignment,
weather shock,
border,
eligibility rule.

Why is the variation plausibly exogenous? What is the argument that treatment variation is unrelated to potential outcomes? Is the argument institutional, statistical, graphical, or theoretical?
What is the comparison group? Who or what provides the missing counterfactual? Are they comparable to the treated units?
What is the identifying assumption? Every design has one. Examples:

random assignment,
parallel trends,
continuity,
exclusion restriction,
no unobserved confounding,
no manipulation,
no spillovers.

What evidence supports the assumption? Look for:

pre-trends,
balance tests,
placebo tests,
robustness checks,
sensitivity analysis,
institutional detail,
graphical diagnostics.

What effect is identified? Is it ATE, ATT, LATE, a local cutoff effect, a cohort-specific effect, or a short-run policy effect? For whom does it apply?
What are the main threats? Could there be sorting, spillovers, anticipation, other simultaneous policies, measurement changes, or endogenous timing?
How precise is the estimate? Are standard errors clustered appropriately? Is there serial correlation? Are there few treated units? Are confidence intervals wide?
Does the result generalize? Would the effect apply to other populations, time periods, institutions, or scales?

8.25 Common mistakes.

Calling any observational variation a natural experiment. Not every policy change or shock is a natural experiment. A natural experiment requires a credible argument that the treatment variation is plausibly exogenous.
Thinking quasi-experimental methods automatically identify causality. Difference-in-differences, RD, IV, and synthetic control are not magic. They identify causal effects only under assumptions.
Ignoring why treatment occurred. Policy adoption is often endogenous. Governments adopt policies in response to economic, political, or social conditions. Ignoring this can bias estimates.
Focusing only on statistical significance. A statistically significant quasi-experimental estimate may still be biased if the design is invalid.
Overgeneralizing local estimates. RD estimates apply near cutoffs. IV estimates apply to compliers. Case-study estimates apply to particular treated units. Local effects should not be casually generalized to everyone.
Treating robustness checks as proof. Robustness checks can support credibility, but they cannot prove identifying assumptions. A flawed design can be robustly wrong.
Ignoring mechanisms. Even if a policy has an effect, understanding the mechanism matters for interpretation, targeting, and scale-up.
Ignoring equilibrium responses. Economic agents respond to policies. Firms, workers, consumers, governments, and markets adjust. A small quasi-experimental estimate may not predict effects at larger scale.

8.26 Application checklist.

Use the following checklist when designing or evaluating a natural experiment.

Define the treatment. What exactly changes? Who is exposed? When does treatment begin? Is treatment binary, continuous, repeated, or variable in intensity?
Define the outcome. What outcome is measured? When is it measured? Could measurement change because of treatment?
Define the estimand. Are you estimating ATE, ATT, LATE, a local cutoff effect, a dynamic effect, or a distributional effect?
Identify the source of variation. What generates treatment variation? Is it a rule, shock, cutoff, lottery, rollout, border, judge assignment, or policy change?
Explain why the variation is credible. Why is treatment variation plausibly unrelated to potential outcomes? Could units manipulate treatment status? Was treatment assigned in response to expected outcomes?
Identify the comparison group. Who represents the missing counterfactual? Why are they credible?
State the identifying assumption. What must be true for the estimate to be causal? State the assumption in plain language and, when possible, mathematically.
Test implications where possible. Use balance tests, pre-trends, placebo outcomes, placebo dates, density tests, and robustness checks.
Consider threats. Look for confounding, sorting, spillovers, anticipation, measurement changes, simultaneous policies, attrition, and heterogeneous effects.
Interpret narrowly and honestly. What effect was identified? For whom? Under what assumptions? Over what time horizon? Does it generalize?

8.27 Summary

Natural experiments and quasi-experimental designs are central to empirical economics because many important treatments cannot be randomized by researchers. A natural experiment occurs when a real-world event, rule, institution, policy, or shock creates treatment variation that is plausibly as-good-as-random for a particular causal question. A quasi-experimental design uses that variation to approximate the logic of an experiment. Common designs include:

difference-in-differences,
regression discontinuity,
instrumental variables,
synthetic control,
event studies,
interrupted time series,
panel fixed effects,
matching and weighting,
border designs,
institutional assignment designs.

Each design constructs a counterfactual differently. Each depends on assumptions. The credibility of a quasi-experiment depends on:

a clearly defined treatment,
a credible comparison group,
a source of plausibly exogenous variation,
explicit identifying assumptions,
evidence supporting those assumptions,
careful attention to threats such as sorting, spillovers, timing, and simultaneous shocks.

The central question is not whether the study uses a fashionable method. The central question is:

What variation identifies the effect, and why does that variation reveal the missing counterfactual?

A strong natural experiment does not eliminate the need for assumptions. It makes the assumptions visible, defensible, and connected to real institutional variation.

9. Identification Strategy

9.1 Why identification is central to empirical economics

Identification is one of the most important ideas in empirical economics. It is also one of the most frequently misunderstood. In ordinary language, researchers often say things like:

This paper identifies the effect of education on earnings.

or:

The policy change provides identifying variation.

or:

The coefficient is identified using variation across cohorts and states.

These statements all point to the same basic issue: why should the empirical comparison be interpreted causally? A statistical estimate is not automatically a causal estimate. A regression coefficient, difference in means, correlation, forecast, or machine-learning prediction may describe patterns in the observed data without answering the counterfactual question of interest. Identification asks whether a causal quantity can be learned from the available data under a stated set of assumptions. For example, suppose we want to estimate the effect of job training on earnings. The causal quantity might be: $ATT = \mathbb{E}[Y_i(1)-Y_i(0) \mid D_i=1]$ where:

$Y_i(1)$ is earnings if worker $i$ receives training,
$Y_i(0)$ is earnings if worker $i$ does not receive training,
$D_i=1$ means worker $i$ actually received training.

The problem is that for trained workers, we observe $Y_i(1)$ but not $Y_i(0)$. The missing term is: $\mathbb{E}[Y_i(0) \mid D_i=1]$ That is, what would trained workers have earned without training? Identification is about whether the research design gives us a credible way to recover this missing counterfactual from observed data. A study is not identified merely because it has data. It is not identified merely because it uses regression. It is not identified merely because the coefficient is statistically significant. It is identified only if the assumptions and design allow the causal estimand to be connected to observable quantities. The central identification question is:

What variation in the data is being used to estimate the causal effect, and why is that variation plausibly unrelated to other causes of the outcome?

This question should guide the entire empirical analysis.

9.2 Identification versus estimation

A key distinction is the difference between identification and estimation. An estimand is the target quantity we want to learn. An estimator is the statistical procedure used to estimate that quantity. An estimate is the numerical value produced by applying the estimator to data. For example, the estimand may be the average treatment effect: $ATE = \mathbb{E}[Y_i(1)-Y_i(0)]$ An estimator might be the difference in sample means: $\hat{\tau} = \frac{1}{N_1}\sum_{i:D_i=1}Y_i - \frac{1}{N_0}\sum_{i:D_i=0}Y_i$ The estimate might be: $\hat{\tau}=2{,}500$ meaning the treated group earned $2,500 more than the untreated group on average. But whether this estimate identifies the causal ATE depends on assumptions. If treatment was randomly assigned, then the difference in means may identify the ATE. If treatment was self-selected, then the difference in means may combine the causal effect with selection bias. So the same estimator can be causal in one design and non-causal in another. This is why identification comes before estimation. Identification asks:

Does this empirical comparison correspond to the causal quantity we want?

Estimation asks:

Given that comparison, how precisely can we estimate it from finite data?

A study can be precisely estimated but poorly identified. For example, with a very large dataset, we may estimate: $Wage_i = \alpha + \beta Education_i + u_i$ and obtain a very small standard error for $\hat{\beta}$. But if education is correlated with ability, family background, school quality, or motivation, then $\hat{\beta}$ may not identify the causal effect of education. Precision does not solve endogeneity. A useful rule is:

Standard errors measure uncertainty about estimation. Identification concerns whether the coefficient means what we claim it means.

Both matter, but they are different problems.

9.3 Three meanings of identification

The word identification is used in several related ways.

Statistical identification. A parameter is statistically identified if it can be uniquely recovered from the population distribution of observed data under the model. Suppose a model implies that the joint distribution of observed variables $(Y,X)$ depends on a parameter $\theta$. If two different values of $\theta$ generate the same observable distribution, then $\theta$ is not identified. Formally, let $P_\theta(Y,X)$ be the distribution of observed data implied by parameter $\theta$. The parameter is identified if: $P_{\theta_1}(Y,X)=P_{\theta_2}(Y,X) \implies \theta_1=\theta_2$ In words: if the observable distribution is the same under two parameter values, then those parameter values must be the same. If different parameters generate the same observed data distribution, the data cannot distinguish between them. Statistical identification is especially important in structural models, latent-variable models, simultaneous-equation models, and models with missing data.
Causal identification. A causal effect is identified if it can be expressed in terms of observed data under stated assumptions. For example, the ATE is: $ATE = \mathbb{E}[Y_i(1)-Y_i(0)]$ This is defined using potential outcomes, not directly observed outcomes. If treatment is randomly assigned, then $D_i \perp (Y_i(1),Y_i(0))$ Under this assumption: $\mathbb{E}[Y_i(1)] = \mathbb{E}[Y_i \mid D_i=1]$ and $\mathbb{E}[Y_i(0)] = \mathbb{E}[Y_i \mid D_i=0]$ Therefore: $ATE = \mathbb{E}[Y_i \mid D_i=1] - \mathbb{E}[Y_i \mid D_i=0]$ The causal effect has been rewritten in terms of observable conditional means. That is causal identification.
Empirical identification strategy. In applied economics, an identification strategy is the design-based argument for why an estimate can be interpreted causally. It answers questions such as:
What source of variation is being used?
Why is that variation plausibly exogenous?
What comparison group provides the counterfactual?
What assumptions are required?
What threats could invalidate the design?
What evidence supports the assumptions?

For example, a weak empirical claim is:

We regress wages on education and controls.

A stronger identification strategy is:

A compulsory schooling reform changed educational attainment for some cohorts but not others. We compare affected and unaffected cohorts across regions and birth years, controlling for region and cohort effects. The identifying assumption is that, absent the reform, affected and unaffected cohorts would have followed comparable earnings trajectories.

The second statement explains the source of variation and the assumptions required for a causal interpretation.

9.4 The anatomy of an identification strategy

A good identification strategy has several components.

A clearly defined causal question. The researcher must first define the causal question. For example:

What is the effect of receiving job training on annual earnings two years later among unemployed workers aged 25 to 45?

This is better than asking:

Does job training work?

A precise causal question specifies:

the treatment,
the outcome,
the unit of analysis,
the population,
the time horizon,
the comparison condition.

Without a clear causal question, identification cannot be evaluated.

A target estimand. The researcher must define what effect is being estimated. Possible estimands include $ATE = \mathbb{E}[Y_i(1)-Y_i(0)]$, $ATT = \mathbb{E}[Y_i(1)-Y_i(0) \mid D_i=1]$, $CATE(x)=\mathbb{E}[Y_i(1)-Y_i(0) \mid X_i=x]$, and $LATE=\mathbb{E}[Y_i(1)-Y_i(0) \mid \text{compliers}]$.

Different research designs may identify different estimands.

An RCT with full compliance may identify the ATE for the experimental sample. An IV design may identify a LATE for compliers. An RD design may identify a local effect at the cutoff. A DiD design may identify an average treatment effect for treated units over a particular post-treatment period. A study should not simply say it estimates “the effect.” It should say what effect, for whom, and under what variation.

A source of identifying variation. The researcher must explain what variation in treatment is being used. Examples:
random assignment,
lottery assignment,
policy timing,
eligibility cutoffs,
geographic borders,
judge assignment,
weather shocks,
cohort exposure,
administrative rules,
instrument-induced variation,
within-unit changes over time.

Not all variation is equally useful. If workers choose whether to attend training, variation in training participation may reflect motivation, ability, desperation, or private information. If a lottery assigns training slots, variation in training offers may be plausibly unrelated to those characteristics. The identification strategy should identify which part of the variation is credible.

A counterfactual comparison. The researcher must state what observed group or pattern represents the missing counterfactual. For example:
In an RCT, the control group represents what would have happened to the treated group without treatment.
In DiD, the control group’s trend represents the treated group’s counterfactual trend.
In RD, units just below the cutoff represent the counterfactual for units just above the cutoff, or vice versa.
In IV, compliers under one instrument state represent the counterfactual for compliers under another instrument state.
In fixed effects, the same unit at other times helps form the comparison.

The comparison group is not automatically valid. It must be justified.

Identifying assumptions. Every identification strategy depends on assumptions. Examples:

Design	Core identifying assumption
RCT	Random assignment is independent of potential outcomes
DiD	Treated and control groups would have followed parallel trends absent treatment
RD	Potential outcomes are continuous at the cutoff
IV	Instrument affects outcome only through treatment and is as-good-as-random
Matching	Treatment is independent of potential outcomes conditional on observables
Fixed effects	Time-varying confounders are absent or controlled
Synthetic control	Weighted control units approximate the treated unit’s untreated path

These assumptions are usually not fully testable. But researchers can provide evidence that makes them more or less plausible.

Threats to identification. A credible study must discuss how the identifying assumptions might fail. Examples:
selection into treatment,
omitted variable bias,
reverse causality,
anticipation effects,
spillovers,
attrition,
manipulation around a cutoff,
weak instruments,
differential pre-trends,
measurement error,
simultaneous policy changes,
composition changes,
treatment effect heterogeneity.

A strong empirical paper does not hide threats. It explains them and evaluates their importance.

Evidence supporting the assumptions. Researchers use several tools to support identification strategies:
institutional details,
randomization checks,
balance tests,
pre-trend tests,
placebo tests,
falsification tests,
sensitivity analysis,
robustness checks,
alternative control groups,
density tests near cutoffs,
first-stage tests for IV,
graphical evidence,
qualitative knowledge of assignment rules.

These tools do not prove assumptions with certainty. They discipline the causal argument.

9.5 A simple example: job training and earnings

Suppose we want to estimate the effect of job training on annual earnings. Let: $D_i=1$ if worker $i$ receives job training, and: $D_i=0$ otherwise. Let: $Y_i(1)$ be earnings if worker $i$ receives training, and: $Y_i(0)$ be earnings if worker $i$ does not receive training. The ATT is: $ATT = \mathbb{E}[Y_i(1)-Y_i(0) \mid D_i=1]$ The observed treated-control difference is: $\mathbb{E}[Y_i \mid D_i=1] - \mathbb{E}[Y_i \mid D_i=0]$ Using potential outcomes, this is: $\mathbb{E}[Y_i(1) \mid D_i=1] - \mathbb{E}[Y_i(0) \mid D_i=0]$ But the ATT requires: $\mathbb{E}[Y_i(1) \mid D_i=1] - \mathbb{E}[Y_i(0) \mid D_i=1]$ The missing counterfactual is: $\mathbb{E}[Y_i(0) \mid D_i=1]$ The identification problem is to recover this missing term. Different identification strategies recover it differently. - Randomized controlled trial. If training slots are randomly assigned among eligible workers, then: $D_i \perp (Y_i(1),Y_i(0))$ The control group identifies what would have happened to the treated group without training.

Difference-in-differences. If trained and untrained workers are observed before and after the program, and if their earnings would have followed parallel trends absent training, then the control group’s change can approximate the treated group’s counterfactual change.
Regression discontinuity. If training eligibility is determined by a cutoff, such as income below a threshold or test score below a threshold, then workers just above and below the cutoff may be comparable.
Instrumental variables. If workers are randomly encouraged to attend training, encouragement can be used as an instrument for participation, provided encouragement affects earnings only through training.
Matching. If workers select into training based only on observed characteristics, then comparing trained and untrained workers with similar observed covariates may identify the effect. The key point is that each strategy answers the same counterfactual problem differently.

9.6 Identification as a mapping from causal quantities to observed data

Formally, identification means expressing a causal quantity in terms of the observed data distribution. The observed data might include: $(Y_i,D_i,X_i)$ where: $Y_i$ is the observed outcome, - $D_i$ is treatment status, - $X_i$ is a vector of observed covariates. The causal quantities involve potential outcomes: $Y_i(1),Y_i(0)$ But we do not observe both potential outcomes for each unit. Identification requires assumptions that allow us to write causal objects using observable objects. For example, suppose we assume unconfoundedness: $(Y_i(1),Y_i(0)) \perp D_i \mid X_i$ and overlap: $0<P(D_i=1 \mid X_i=x)<1$ for relevant $x$. Then: $\mathbb{E}[Y_i(1) \mid X_i=x] = \mathbb{E}[Y_i \mid D_i=1,X_i=x]$ and: $\mathbb{E}[Y_i(0) \mid X_i=x] = \mathbb{E}[Y_i \mid D_i=0,X_i=x]$ Therefore: $ATE = \mathbb{E}_X\left[ \mathbb{E}[Y_i \mid D_i=1,X_i] - \mathbb{E}[Y_i \mid D_i=0,X_i] \right]$ This formula identifies the ATE under conditional independence and overlap.

The formula itself is not a research design. The credibility depends on whether the assumptions are believable.

9.7 The role of assumptions

All causal identification depends on assumptions. There is no assumption-free causal inference. Even randomized experiments require assumptions:

treatment is actually randomized,
randomization is implemented correctly,
no differential attrition,
no interference or spillovers,
treatment and control conditions are well defined,
outcomes are measured consistently,
the sample represents the population of interest if external validity is claimed.

Observational studies require even more substantive assumptions. This does not mean causal inference is arbitrary. It means assumptions must be explicit and evaluated. A useful distinction is between statistical assumptions and identifying assumptions. Statistical assumptions might include:

independent sampling,
homoskedasticity,
correct standard error clustering,
distributional assumptions,
large-sample approximations.

Identifying assumptions might include:

no omitted confounding,
parallel trends,
exclusion restriction,
continuity at a cutoff,
no manipulation,
no anticipation,
no spillovers.

A study can satisfy statistical assumptions but fail identifying assumptions. For example, the standard errors in an OLS regression may be correctly clustered, but the coefficient may still be biased by omitted variables. The hierarchy is:

Define the causal estimand.
Establish identification under assumptions.
Estimate the identified quantity.
Conduct inference about sampling uncertainty.

Skipping step 2 is one of the most common mistakes in applied work.

9.8 Exogenous variation

Empirical economists often say they are looking for exogenous variation. Exogenous variation is variation in treatment that is unrelated to unobserved determinants of the outcome. In a regression model: $Y_i = \alpha + \beta D_i + u_i$ we need: $\mathbb{E}[u_i \mid D_i]=0$ or at least: $\operatorname{Cov}(D_i,u_i)=0$ for $\beta$ to have a causal interpretation in the simplest linear model. In potential outcomes language, exogenous treatment assignment often means: $D_i \perp (Y_i(1),Y_i(0))$ or conditionally: $D_i \perp (Y_i(1),Y_i(0)) \mid X_i$ But in applied economics, treatment is often chosen by people, firms, governments, or institutions. That means variation in treatment may reflect unobserved factors.

Examples:

Students choose education partly based on ability and family background.
Firms choose technology adoption partly based on expected demand.
Cities choose policing levels partly based on crime conditions.
Governments choose stimulus spending partly based on economic distress.
Patients receive treatment partly based on illness severity.

In these cases, raw treatment variation is endogenous. Identification strategies search for a subset or source of treatment variation that is plausibly exogenous. For example:

A lottery creates exogenous variation in program offers.
A cutoff creates quasi-exogenous variation near the threshold.
A policy rollout creates variation across time and place.
Weather shocks create variation in agricultural income.
Judge assignment creates variation in sentencing severity.
Draft lotteries create variation in military service.

The key is not whether the treatment itself is generally endogenous. The key is whether the particular variation used for identification is plausibly exogenous.

9.9 Identifying variation versus control variation

Not all variation used in a regression identifies the causal effect. Consider: $Y_{it}=\alpha_i+\lambda_t+\beta D_{it}+u_{it}$ where:

$\alpha_i$ are unit fixed effects,
$\lambda_t$ are time fixed effects,
$D_{it}$ is treatment.

The coefficient $\beta$ is identified from variation in $D_{it}$ within units over time, after removing common time shocks. This is different from cross-sectional variation across units. For example, suppose $i$ indexes states and $D_{it}$ is a minimum wage policy. With state fixed effects and year fixed effects, identification comes from states that change their minimum wage relative to their own baseline and relative to national time patterns. The question is then:

Are the timing and magnitude of state minimum wage changes plausibly unrelated to other state-specific shocks affecting employment?

If states raise minimum wages during local booms, the estimate may be biased upward. If they raise minimum wages during local downturns, the estimate may be biased downward. A regression specification does not by itself guarantee credible identifying variation. The researcher must explain what variation remains after controls and why that variation is credible. This is why applied economists often ask:

Where is the coefficient coming from?

or:

What comparison identifies the estimate?

9.10 Identification in randomized experiments

Randomized experiments provide the cleanest identification strategy because treatment assignment is controlled by the researcher. Let $D_i$ be randomly assigned. Then: $D_i \perp (Y_i(1),Y_i(0))$ This implies: $\mathbb{E}[Y_i(0) \mid D_i=1] = \mathbb{E}[Y_i(0) \mid D_i=0]$ and: $\mathbb{E}[Y_i(1) \mid D_i=1] = \mathbb{E}[Y_i(1) \mid D_i=0]$ The observed control group identifies the treated group’s missing untreated counterfactual. The difference in means identifies the ATE: $ATE = \mathbb{E}[Y_i \mid D_i=1] - \mathbb{E}[Y_i \mid D_i=0]$ under full compliance and no spillovers.

However, the identification strategy still needs to address implementation problems. Possible threats include:

noncompliance,
attrition,
spillovers,
imperfect treatment implementation,
measurement error,
differential survey response,
Hawthorne effects,
small sample imbalance,
multiple testing,
limited external validity.

Randomization is powerful because it directly addresses confounding. But it does not eliminate the need for careful design and interpretation.

9.11 Identification in difference-in-differences

Difference-in-differences uses changes over time in treated and control groups. Suppose there are two groups and two periods. The treated group receives treatment after the first period. The control group does not. The DiD estimand is: $\left(\mathbb{E}[Y_{T,after}]-\mathbb{E}[Y_{T,before}]\right) - \left(\mathbb{E}[Y_{C,after}]-\mathbb{E}[Y_{C,before}]\right)$ The identifying assumption is parallel trends: > In the absence of treatment, the treated and control groups would have experienced the same average change in outcomes. In potential outcomes notation: $\mathbb{E}[Y_{T,after}(0)-Y_{T,before}(0)] = \mathbb{E}[Y_{C,after}(0)-Y_{C,before}(0)]$ This assumption allows the control group’s change to stand in for the treated group’s missing counterfactual change. The treated group’s observed change is: $\mathbb{E}[Y_{T,after}(1)-Y_{T,before}(0)]$ The missing counterfactual is: $\mathbb{E}[Y_{T,after}(0)]$ DiD identifies the effect if the control group provides a credible estimate of what would have happened to the treated group after treatment absent the policy.

Threats include:

differential pre-trends,
simultaneous policies,
anticipation effects,
spillovers,
changing composition,
endogenous treatment timing,
heterogeneous treatment effects under staggered adoption.

Evidence supporting DiD often includes:

pre-treatment trend graphs,
event studies,
placebo treatment dates,
alternative control groups,
covariate balance over time,
institutional explanation for policy timing.

9.12 Identification in regression discontinuity

Regression discontinuity uses a cutoff rule that changes treatment assignment. Let $R_i$ be a running variable and $c$ be a cutoff. In a sharp RD: $D_i = 1(R_i \geq c)$ or: $D_i = 1(R_i < c)$ The RD estimand is the discontinuous jump in the outcome at the cutoff: $\tau_{RD} = \lim_{r \downarrow c}\mathbb{E}[Y_i \mid R_i=r] - \lim_{r \uparrow c}\mathbb{E}[Y_i \mid R_i=r]$ with the direction depending on which side receives treatment. The identifying assumption is that potential outcomes are continuous at the cutoff: $\lim_{r \downarrow c}\mathbb{E}[Y_i(0) \mid R_i=r] = \lim_{r \uparrow c}\mathbb{E}[Y_i(0) \mid R_i=r]$ and similarly for $Y_i(1)$ when relevant.

In words:

Units just above and just below the cutoff would have had similar outcomes absent the treatment.

The cutoff creates local quasi-random variation in treatment. Threats include:

precise manipulation of the running variable,
sorting around the cutoff,
other policies changing at the same cutoff,
discontinuities in covariates,
inappropriate bandwidth choices,
misspecified functional form.

Evidence supporting RD includes:

graphs of outcomes around the cutoff,
density tests for bunching,
covariate balance near the cutoff,
placebo cutoffs,
bandwidth sensitivity,
institutional details about the assignment rule.

RD identifies a local effect near the cutoff. It may not generalize to units far from the cutoff.

9.13 Identification in instrumental variables

Instrumental variables are used when treatment is endogenous. Suppose: $Y_i = \alpha + \beta D_i + u_i$ and: $\operatorname{Cov}(D_i,u_i) \neq 0$ An instrument $Z_i$ provides variation in $D_i$ that is plausibly unrelated to $u_i$. The core IV assumptions are: Relevance. The instrument affects treatment: $\operatorname{Cov}(Z_i,D_i) \neq 0$Independence. The instrument is as-good-as-random with respect to potential outcomes: $Z_i \perp (Y_i(1),Y_i(0))$ or conditionally independent given covariates. Exclusion restriction. The instrument affects the outcome only through treatment. In causal graph terms: $Z_i \rightarrow D_i \rightarrow Y_i$ but no direct path: $Z_i \rightarrow Y_i$ except through $D_i$. - Monotonicity. The instrument moves treatment in the same direction for all units. It rules out defiers. Under these assumptions, IV identifies a local average treatment effect for compliers: $LATE = \mathbb{E}[Y_i(1)-Y_i(0) \mid \text{compliers}]$ The Wald estimand for a binary instrument and binary treatment is: $\tau_{IV} = \frac{\mathbb{E}[Y_i \mid Z_i=1]-\mathbb{E}[Y_i \mid Z_i=0]} {\mathbb{E}[D_i \mid Z_i=1]-\mathbb{E}[D_i \mid Z_i=0]}$ The numerator is the reduced-form effect of the instrument on the outcome.

The denominator is the first-stage effect of the instrument on treatment. Threats include:

weak first stage,
violation of exclusion restriction,
direct effects of the instrument,
instrument correlated with omitted variables,
heterogeneous effects and unclear complier population,
violations of monotonicity.

The hardest part of IV is usually not estimation. It is defending the exclusion restriction.

9.14 Identification in fixed effects designs

Fixed effects use repeated observations to control for stable unobserved differences across units. A common model is: $Y_{it}=\alpha_i+\lambda_t+\beta D_{it}+u_{it}$ where:

$\alpha_i$ is a unit fixed effect,
$\lambda_t$ is a time fixed effect,
$D_{it}$ is treatment,
$u_{it}$ is the remaining error.

The unit fixed effect controls for all time-invariant characteristics of unit $i$. For example, if $i$ indexes workers, $\alpha_i$ controls for stable worker traits such as baseline ability, early family background, or persistent personality traits. If $i$ indexes firms, $\alpha_i$ controls for stable firm quality, culture, location, or long-run productivity differences. The time fixed effect controls for shocks common to all units in a period. The coefficient $\beta$ is identified from within-unit changes in treatment over time. The identifying assumption is not that treatment is randomly assigned. It is that, after removing unit and time fixed effects, changes in treatment are unrelated to remaining time-varying unobserved determinants of the outcome. Formally, we need something like: $\mathbb{E}[u_{it} \mid D_{i1},D_{i2},\dots,D_{iT},\alpha_i,\lambda_t]=0$ or a weaker condition appropriate to the design.

Fixed effects do not solve all confounding. They remove time-invariant confounders, but not time-varying confounders. For example, if firms adopt new technology when demand is rising, firm fixed effects do not solve the problem. The adoption timing is still endogenous. Threats include:

time-varying confounding,
reverse causality,
dynamic feedback,
measurement error worsened by differencing,
anticipation,
treatment effect heterogeneity,
limited within-unit variation.

Fixed effects are useful, but they are not magic.

9.15 Identification in matching and weighting

Matching and weighting try to construct comparable treated and untreated groups using observed covariates. The key identifying assumption is unconfoundedness: $(Y_i(1),Y_i(0)) \perp D_i \mid X_i$ This says that conditional on observed characteristics $X_i$, treatment assignment is independent of potential outcomes. The second key assumption is overlap: $0<P(D_i=1 \mid X_i=x)<1$ for relevant $x$.

Under these assumptions, we can compare treated and untreated units with the same covariates. Matching directly pairs or groups similar units. Weighting reweights observations to make covariate distributions comparable. For example, inverse probability weighting uses the propensity score: $e(X_i)=P(D_i=1 \mid X_i)$ A common weighting expression for the ATE is: $\mathbb{E}\left[\frac{D_iY_i}{e(X_i)} - \frac{(1-D_i)Y_i}{1-e(X_i)}\right]$ Matching and weighting can be powerful when selection occurs on observed variables.

But they do not solve selection on unobservables. If job training participation depends on motivation and motivation is unmeasured, matching on age, education, and prior earnings may still leave bias. Threats include:

omitted unobserved confounders,
poor overlap,
high-dimensional covariates,
model dependence,
bad control variables,
post-treatment covariates,
extreme weights.

Matching and weighting improve comparability on measured variables. They do not create randomization unless the unconfoundedness assumption is credible.

9.16 Identification in synthetic control designs

Synthetic control is often used for case studies with one or a small number of treated units. Example:

one state adopts a policy,
one country experiences a reform,
one city hosts a major event,
one region experiences a shock.

The method constructs a weighted combination of untreated units that resembles the treated unit before treatment. Let unit $1$ be treated and units $2,\dots,J+1$ be controls. A synthetic control chooses weights $w_j$ such that: $\sum_{j=2}^{J+1} w_j X_j \approx X_1$ and: $\sum_{j=2}^{J+1} w_j Y_{jt} \approx Y_{1t}$ for pre-treatment periods. The post-treatment effect is estimated as: $\hat{\tau}_t = Y_{1t}-\sum_{j=2}^{J+1}w_jY_{jt}$ for post-treatment periods $t$.

The identifying assumption is that the synthetic control approximates the treated unit’s untreated counterfactual path. This is more credible when:

pre-treatment fit is strong,
no other major shocks affect the treated unit at the treatment time,
control units are not affected by spillovers,
the treated unit can plausibly be represented as a weighted combination of controls.

Threats include:

poor pre-treatment fit,
spillovers to control units,
simultaneous shocks,
subjective specification choices,
limited donor pool,
extrapolation beyond available controls.

Synthetic control makes the counterfactual visible: the weighted comparison path is the estimated no-treatment path.

9.17 Identification and DAGs

DAGs help clarify identification by showing which paths create causal and non-causal associations. Suppose the causal graph is: $Ability \rightarrow Education \rightarrow Wage$ and: $Ability \rightarrow Wage$ The causal path of interest is: $Education \rightarrow Wage$ The backdoor path is: $Education \leftarrow Ability \rightarrow Wage$ If ability is unobserved and not controlled, the effect of education is confounded.

A valid adjustment set blocks all non-causal backdoor paths without blocking the causal path or opening collider paths. If ability were observed, controlling for ability could identify the causal effect under suitable assumptions. But suppose we instead control for occupation: $Education \rightarrow Occupation \rightarrow Wage$ Occupation is a mediator. Controlling for it blocks part of the causal effect of education. Or suppose we control for admission to an elite program: $Talent \rightarrow Admission \leftarrow Luck$ Conditioning on admission may create collider bias.

DAGs do not identify effects by themselves. They represent assumptions about causal structure. Given those assumptions, they help determine whether a causal effect is identifiable and what variables should or should not be controlled. DAGs are useful because they make identification assumptions explicit.

9.18 Identification and bad controls

A common mistake is to think that adding more control variables always improves identification. This is false. Controls help only when they block non-causal sources of association without blocking causal effects or opening new biasing paths. Bad controls include:

mediators,
colliders,
descendants of colliders,
post-treatment variables,
variables affected by treatment,
variables that are themselves outcomes.

For example, suppose: $Education \rightarrow Occupation \rightarrow Wage$ If the goal is the total effect of education on wages, controlling for occupation is bad because occupation is part of the mechanism. Suppose: $Treatment \rightarrow Health \leftarrow Baseline\ Risk$ If health is affected by treatment and by baseline risk, conditioning on health can create a non-causal association between treatment and baseline risk. Suppose: $Treatment \rightarrow Employment \rightarrow Income$ If we control for employment when estimating the effect of treatment on income, we may remove part of the effect.

The right control set depends on the estimand. If the goal is the total effect, do not control for mediators. If the goal is a direct effect, controlling for mediators may be appropriate, but the interpretation changes and additional assumptions are needed. Identification is not about maximizing the number of controls. It is about choosing controls based on causal logic.

9.19 Local versus global identification

Many research designs identify local effects. A local effect applies to a specific subset of units, margin, place, time, or source of variation. Examples:

Regression discontinuity. RD identifies the effect for units near the cutoff. A scholarship RD at GPA 3.5 identifies the effect for students near 3.5, not necessarily for students with GPA 2.0 or 4.0.
Instrumental variables. IV often identifies the effect for compliers. If distance to college is an instrument for college attendance, the estimate applies to people whose college attendance changes because of distance. It may not apply to people who would always attend or never attend regardless of distance.
Difference-in-differences. DiD often identifies effects for treated units affected by the policy change during the studied period. The estimate may not apply to other places or later scale-ups.
RCTs. An RCT identifies the effect for the experimental sample under the implemented version of treatment. Generalizing beyond that sample requires external validity assumptions. A rigorous interpretation should therefore state:
whose effect is identified,
what margin of treatment variation identifies it,
what time horizon it covers,
what population it may generalize to,
what population it may not generalize to.

A common error is to estimate a local effect and then describe it as a universal effect.

9.20 Identification and heterogeneity

Treatment effects are often heterogeneous. That means: $Y_i(1)-Y_i(0)$ may differ across units.

When treatment effects are heterogeneous, the source of identifying variation matters because different designs weight units differently. For example, an IV estimate weights compliers affected by the instrument. An RD estimate focuses on units near the cutoff. A DiD estimate may weight groups and time periods in complicated ways, especially with staggered adoption and heterogeneous effects. An RCT estimate may represent the average effect for the experimental sample, but not necessarily for the target population. Therefore, when effects are heterogeneous, identification requires asking:

Which treatment effects are being averaged, and with what weights?

This is not a minor detail. It can change the substantive interpretation. For example, a job training program may have large effects for young displaced workers and small effects for older workers. An estimate based mostly on young compliers should not be interpreted as the effect for all unemployed workers. Heterogeneity connects identification to external validity and policy relevance.

9.21 Identification and external validity

Internal validity asks whether the study identifies a causal effect in the study setting. External validity asks whether that effect generalizes to other settings. A study may be internally valid but externally limited. For example:

An RD design may identify the effect near a cutoff but not far from it.
An IV design may identify a complier effect but not an average effect for everyone.
An RCT may identify the effect in one city but not another.
A policy evaluation during a recession may not apply during an expansion.
A small pilot may not predict effects at national scale.

External validity is not automatic. It requires its own assumptions. To evaluate external validity, ask:

Is the population similar?
Is the treatment version similar?
Is implementation quality similar?
Are institutions similar?
Are baseline conditions similar?
Are there scale effects?
Are there equilibrium effects?
Are there spillovers in the new setting?
Are treatment effects heterogeneous across relevant groups?

Identification of the original effect does not guarantee transportability.

9.22 How researchers defend identification strategies

Researchers defend identification strategies using several forms of evidence.

Institutional knowledge. Institutional details explain how treatment was assigned. Examples:
Was a lottery actually random?
How was a cutoff enforced?
Why did some regions adopt a policy earlier?
How were judges assigned to cases?
Who controlled program eligibility?

Institutional knowledge often matters more than statistical complexity.

Balance tests. Balance tests compare pre-treatment characteristics across treated and control groups. In an RCT, balance tests check whether randomization produced similar groups. In RD, covariate balance near the cutoff supports the claim that units near the threshold are comparable. In IV, balance across instrument values supports independence. Balance tests cannot prove identification, especially for unobserved variables. But imbalance can reveal problems.
Pre-trend tests. Pre-trend tests are especially important in DiD and event studies. They ask whether treated and control groups followed similar trends before treatment. If they diverged before treatment, parallel trends is less plausible. However, passing a pre-trend test does not prove future parallel trends. It only provides supporting evidence.
Placebo tests. A placebo test applies the design where no effect should be found. Examples:
use a fake treatment date before the actual policy,
test outcomes that should not be affected,
apply the method to groups not exposed to treatment,
test cutoffs where no policy exists.

If the design finds effects where none should exist, the identification strategy is suspect.

Falsification tests. Falsification tests examine implications of the identifying assumptions. For example:
In RD, covariates should not jump at the cutoff.
In IV, the instrument should not predict pre-treatment outcomes.
In DiD, treatment should not affect outcomes before treatment occurs.

Falsification tests do not prove assumptions, but they can reveal violations.

Robustness checks. Robustness checks ask whether results persist under reasonable alternative specifications. Examples:
alternative control groups,
alternative bandwidths,
alternative functional forms,
alternative time windows,
alternative covariate sets,
alternative outcome definitions,
alternative clustering choices,
excluding influential observations.

Robustness is not a substitute for identification. But a result that disappears under minor reasonable changes may be less credible.

Sensitivity analysis. Sensitivity analysis asks how strong a violation of assumptions would need to be to change the conclusion. For example:
How strong would an unobserved confounder need to be to explain the estimate?
How much differential attrition would overturn the result?
How large would spillovers need to be?
How sensitive is the estimate to weak instruments?

Sensitivity analysis is useful because identifying assumptions are rarely certain.

9.23 What identification cannot do

Identification is powerful, but it has limits.

Identification does not eliminate uncertainty. Even if a causal effect is identified, estimates are subject to sampling error. A valid design still needs standard errors, confidence intervals, and careful inference.
Identification does not guarantee external validity. A well-identified local effect may not generalize.
Identification does not guarantee policy relevance. A causal effect may be too small, too costly, too delayed, or too narrowly targeted to matter for policy.
Identification does not fix bad measurement. If the treatment or outcome is poorly measured, identification may be undermined.
Identification does not make assumptions true. A research design depends on assumptions. Naming a method does not guarantee those assumptions hold. For example:
Calling a study DiD does not prove parallel trends.
Calling a variable an instrument does not prove exclusion.
Calling a design RD does not prove no manipulation.
Including fixed effects does not eliminate time-varying confounding.

Methods are only as credible as their assumptions.

9.24 Common mistakes about identification.

Equating regression with identification. Regression is an estimation tool. It is not an identification strategy by itself. The question is what variation in the regression identifies the coefficient and whether that variation is plausibly causal.
Believing controls solve everything. Controls help only if they block confounding without introducing new bias. Omitted unobservables, post-treatment controls, mediators, and colliders can all create problems.
Confusing statistical significance with identification. A statistically significant coefficient can be badly biased. Significance answers a sampling question, not a causal identification question.
Ignoring the estimand. Researchers sometimes report an estimate without explaining whether it represents ATE, ATT, LATE, a local cutoff effect, or something else. The estimand determines the interpretation.
Ignoring the source of variation. A coefficient may combine many sources of variation. Some may be credible; others may not. A good study explains which variation is doing the identifying work.
Overgeneralizing local estimates. An RD estimate near a cutoff, an IV estimate for compliers, or an RCT estimate in one setting should not automatically be generalized to everyone.
Treating robustness checks as proof. Robustness checks can support a design, but they do not prove identifying assumptions.
Hiding assumptions. Every causal estimate depends on assumptions. The best empirical work states them clearly.

9.25 Application checklist.

When evaluating or designing an empirical study, use the following checklist.

State the causal question. What is the intervention? What is the outcome? Who is the population? What is the time horizon?
Define the estimand. Are you estimating ATE, ATT, ATU, CATE, LATE, a local cutoff effect, a dynamic effect, or a distributional effect?
Identify the missing counterfactual. For the treated group, what untreated outcome is missing? For the untreated group, what treated outcome is missing?
Identify the source of variation. What variation in treatment is being used? Is it random assignment, policy timing, cutoff-based variation, instrument-induced variation, within-unit variation, or selection on observables?
Explain why the variation is credible. Why is this variation plausibly unrelated to other determinants of the outcome? What institutional details support this claim?
State the identifying assumptions. Examples:

random assignment,
parallel trends,
exclusion restriction,
no manipulation,
unconfoundedness,
overlap,
no time-varying confounding,
no interference.

Assess threats. Could the assumptions fail because of selection, omitted variables, reverse causality, spillovers, anticipation, attrition, manipulation, measurement error, or simultaneous shocks?
Provide supporting evidence. Use balance tests, pre-trends, placebo tests, falsification tests, robustness checks, graphical analysis, and institutional details.
Interpret the estimate carefully. What effect was identified? For whom? Using what variation? Over what time horizon? Under what assumptions?
Discuss external validity. Would the effect generalize to other populations, places, times, policy versions, or scales?

9.26 Summary

Identification is the logic that connects a causal question to observed data. A causal estimand, such as: $ATE = \mathbb{E}[Y_i(1)-Y_i(0)]$ is defined in terms of potential outcomes. But potential outcomes are not all observed. Identification asks whether the missing counterfactuals can be recovered from observed data under credible assumptions.

The central question is:

What variation identifies the effect, and why is that variation plausibly exogenous?

Different research designs answer this question differently. Randomized experiments use random assignment. Difference-in-differences uses counterfactual trends. Regression discontinuity uses local variation around a cutoff. Instrumental variables use instrument-induced variation. Fixed effects use within-unit changes over time. Matching and weighting use comparisons among units with similar observed covariates. Synthetic control constructs a weighted comparison unit. No method is automatically causal. Each depends on assumptions. A rigorous identification strategy defines the causal question, estimand, source of variation, comparison group, identifying assumptions, threats to validity, and scope of interpretation. The core lesson is:

Identification is not a statistical trick. It is the argument that the empirical comparison recovers the relevant counterfactual.

10. Instrumental Variables

10.1 Why instrumental variables are needed

Instrumental variables, usually abbreviated as IV, are used when the treatment or explanatory variable of interest is endogenous. The basic problem is this. Suppose we want to estimate the causal effect of $X_i$ on $Y_i$ using the regression: $Y_i = \alpha + \beta X_i + u_i$ The coefficient $\beta$ has a causal interpretation only if variation in $X_i$ is unrelated to the unobserved determinants of $Y_i$ contained in $u_i$. A key exogeneity condition is: $\mathbb{E}[u_i \mid X_i] = 0$ or, in a weaker linear form: $\operatorname{Cov}(X_i,u_i)=0$ If this condition fails, ordinary least squares does not identify the causal effect of $X_i$ on $Y_i$.

Endogeneity can arise from many sources:

omitted variables,
reverse causality,
simultaneity,
selection into treatment,
measurement error,
dynamic feedback,
equilibrium behavior,
model misspecification.

For example, suppose we estimate the effect of education on wages: $Wage_i = \alpha + \beta Education_i + u_i$ Education may be endogenous because people with more education may differ from people with less education in ability, family background, motivation, school quality, parental networks, local labor markets, and expectations. If these factors affect wages and are not fully observed, then they enter $u_i$ and may be correlated with $Education_i$.

In that case: $\operatorname{Cov}(Education_i,u_i) \neq 0$ OLS may estimate a relationship between education and wages, but that relationship may not equal the causal return to education.

Instrumental variables provide a possible solution. The idea is to find a variable $Z_i$ that changes $X_i$ but is otherwise unrelated to the unobserved determinants of $Y_i$. An instrument creates a source of variation in treatment that is more plausibly exogenous than the treatment itself. The goal is not to use all variation in $X_i$. The goal is to isolate the part of $X_i$ that is caused by $Z_i$ and use only that variation to estimate the effect of $X_i$ on $Y_i$.

10.2 The basic IV idea

Suppose we want the causal effect of $X_i$ on $Y_i$: $Y_i = \alpha + \beta X_i + u_i$ but $X_i$ is endogenous: $\operatorname{Cov}(X_i,u_i) \neq 0$ An instrument $Z_i$ is a variable that satisfies two core conditions: 1. Relevance: $Z_i$ affects $X_i$. 2. Exclusion: $Z_i$ affects $Y_i$ only through $X_i$. Relevance means: $\operatorname{Cov}(Z_i,X_i) \neq 0$ Exclusion means that $Z_i$ has no direct effect on $Y_i$ except through $X_i$. A simple DAG for a valid instrument is: $Z \rightarrow X \rightarrow Y$ There should be no direct arrow: $Z \rightarrow Y$ and no open backdoor path connecting $Z$ to $Y$ through omitted causes.

The instrument gives us a piece of variation in $X$ that can be interpreted as exogenous. We then ask whether the part of $X$ shifted by $Z$ affects $Y$. For example, suppose we want to estimate the effect of college attendance on earnings. College attendance is endogenous because students who attend college may differ from those who do not. An instrument might be distance to the nearest college, if living closer to a college makes attendance more likely but does not affect earnings through any channel other than college attendance. The proposed causal chain is: $Distance\ to\ College \rightarrow College\ Attendance \rightarrow Earnings$ This instrument is relevant if distance to college actually affects college attendance.

It satisfies exclusion only if distance to college affects earnings solely through college attendance. That is a strong assumption. It may fail if people who live near colleges differ systematically from people who live far away in labor markets, family background, urbanization, school quality, or local economic opportunity. This example illustrates the central feature of IV: relevance can often be tested, but exclusion usually must be defended using theory, institutional knowledge, and indirect evidence.

10.3 IV as a solution to omitted variable bias

Consider the true model: $Y_i = \alpha + \beta X_i + \gamma A_i + \varepsilon_i$ where $A_i$ is an omitted variable such as ability. Suppose we estimate: $Y_i = \alpha + \beta X_i + u_i$ where: $u_i = \gamma A_i + \varepsilon_i$ If $A_i$ is correlated with $X_i$, then $X_i$ is endogenous: $\operatorname{Cov}(X_i,u_i) \neq 0$ An instrument $Z_i$ can help if it shifts $X_i$ but is unrelated to $A_i$ and $\varepsilon_i$.

The structure is:

\[Z_i \rightarrow X_i \rightarrow Y_i\]

with omitted confounding:

\[A_i \rightarrow X_i\]

\[A_i \rightarrow Y_i\]

The instrument must not be connected to $A_i$: $Z_i \perp A_i$ at least in the relevant causal sense. For education and wages, a compulsory schooling law may serve as an instrument if it increases schooling for some individuals but is otherwise unrelated to their ability, motivation, or family background. The idea is that the law creates variation in schooling that is not chosen by individuals. That law-induced variation may be more credible for causal inference than ordinary differences in schooling. The IV estimate then uses only the schooling variation generated by the compulsory schooling law, not all schooling variation. This is both the strength and limitation of IV. It can isolate plausibly exogenous variation, but the resulting estimate may apply only to the people whose behavior was changed by the instrument.

10.4 The two core IV assumptions

Relevance. The instrument must affect the endogenous treatment or explanatory variable. Formally: $\operatorname{Cov}(Z_i,X_i) \neq 0$ In a regression framework, the first-stage relationship is: $X_i = \pi_0 + \pi_1 Z_i + v_i$ Relevance requires: $\pi_1 \neq 0$ If $Z_i$ does not predict $X_i$, then $Z_i$ cannot be used to isolate variation in $X_i$. Examples of relevance: Examples/considerations: A compulsory schooling law must actually increase years of schooling; A randomly assigned encouragement must actually increase program participation; Eligibility at a cutoff must actually increase treatment receipt; Distance to a college must actually affect college attendance; Judge assignment must actually affect sentencing severity. Relevance is empirically testable because both $Z_i$ and $X_i$ are observed. Researchers usually report the first-stage coefficient and first-stage strength. A weak first stage creates weak instrument problems, discussed later in this section.
Exclusion restriction. The exclusion restriction says that the instrument affects the outcome only through the treatment. In a DAG, the valid path is: $Z \rightarrow X \rightarrow Y$ There should be no direct causal path: $Z \rightarrow Y$ and no indirect path from $Z$ to $Y$ except through $X$. For example, suppose distance to college is used as an instrument for college attendance. The exclusion restriction says that distance to college affects earnings only because it affects college attendance. This may fail if distance to college also captures: Examples/considerations: urban versus rural location,; local labor market quality,; parental education,; neighborhood income,; school quality,; access to professional networks,; migration opportunities. If any of these affect earnings, then distance to college may have a path to earnings not mediated by college attendance. The exclusion restriction is usually the most controversial IV assumption. It cannot generally be tested directly because it is a claim about unobserved counterfactual pathways. Researchers defend the exclusion restriction using: Examples/considerations: institutional details,; historical context,; balance tests,; placebo outcomes,; robustness checks,; alternative instruments,; falsification tests,; sensitivity analysis,; clear discussion of possible violations. A strong first stage is not enough. An instrument can strongly predict treatment and still be invalid if it affects the outcome through other channels.

10.5 Additional IV assumptions

The two-assumption summary of IV as relevance plus exclusion is useful, but rigorous causal interpretation usually requires additional assumptions.

Independence. Independence means the instrument is as-good-as-random with respect to potential outcomes. For binary treatment, a common statement is: $Z_i \perp \big(Y_i(1),Y_i(0)\big)$ This says the instrument is independent of the outcomes each unit would have under treatment and control. For example, in a lottery-based instrument, random lottery assignment may be independent of potential outcomes by design. In observational IV settings, independence is an assumption. It may be plausible if the instrument comes from a rule, shock, or institutional process unrelated to individual potential outcomes. Examples: Examples/considerations: random draft lottery numbers,; random judge assignment,; randomized encouragement,; arbitrary eligibility rules,; weather shocks in some agricultural settings. Independence may fail if the instrument is chosen, anticipated, manipulated, or correlated with omitted determinants of the outcome.
Monotonicity. Monotonicity is important for interpreting IV as a Local Average Treatment Effect. For a binary instrument $Z_i \in \{0,1\}$ and binary treatment $D_i \in \{0,1\}$, define potential treatment statuses: $D_i(1)$ and $D_i(0)$ where: Examples/considerations: $D_i(1)$ is whether unit $i$ receives treatment when $Z_i=1$,; $D_i(0)$ is whether unit $i$ receives treatment when $Z_i=0$. Monotonicity requires: $D_i(1) \geq D_i(0)$ for every unit, or the reverse inequality for every unit depending on the direction of the instrument. In words: “The instrument moves treatment in one direction for everyone.” It rules out defiers. If $Z_i$ is an encouragement to participate in job training, monotonicity says the encouragement may cause some people to participate who otherwise would not, but it does not cause anyone to avoid training who otherwise would have participated. This is plausible in many encouragement designs but not automatic.
Stable unit treatment value and no interference. As with other causal designs, IV usually requires some version of SUTVA. The treatment must be well defined, and one unit’s instrument or treatment should not affect another unit’s outcome unless the analysis explicitly models spillovers. This may fail in general equilibrium settings. For example, a job training instrument may increase employment for treated workers but reduce job opportunities for untreated workers if jobs are scarce. In that case, untreated workers’ outcomes may be affected by others’ treatment status.

10.6 Reduced form, first stage, and IV estimand

A useful way to understand IV is to break it into three relationships:

the first stage,
the reduced form,
the IV ratio.

Suppose $Z_i$ is a binary instrument, $D_i$ is a binary treatment, and $Y_i$ is the outcome. The first stage is the effect of the instrument on treatment: $\mathbb{E}[D_i \mid Z_i=1] - \mathbb{E}[D_i \mid Z_i=0]$ The reduced form is the effect of the instrument on the outcome: $\mathbb{E}[Y_i \mid Z_i=1] - \mathbb{E}[Y_i \mid Z_i=0]$ The IV estimand is the ratio: $\beta_{IV} = \frac{\mathbb{E}[Y_i \mid Z_i=1] - \mathbb{E}[Y_i \mid Z_i=0]} {\mathbb{E}[D_i \mid Z_i=1] - \mathbb{E}[D_i \mid Z_i=0]}$ This ratio asks:

How much did the outcome change because of the instrument, divided by how much the treatment changed because of the instrument?

If the instrument affects the outcome only by changing treatment, then the reduced-form effect should operate through the first-stage effect. For example, suppose a randomized training encouragement increases training participation by 20 percentage points and increases average earnings by $800. The IV estimate is: $\frac{800}{0.20}=4{,}000$ This suggests that the effect of training for those induced into training by the encouragement is $4,000. The reduced form is policy-relevant by itself because it estimates the effect of the instrument or offer. The IV ratio rescales that effect into the effect of actual treatment among compliers.

10.7 Two-stage least squares

The most common regression implementation of IV is two-stage least squares, abbreviated 2SLS. Suppose the structural equation is: $Y_i = \alpha + \beta X_i + u_i$ where $X_i$ is endogenous. Let $Z_i$ be an instrument.

First stage. The first stage regresses the endogenous variable on the instrument: $X_i = \pi_0 + \pi_1 Z_i + v_i$ This produces predicted values: $\widehat{X}_i$ The predicted values capture the component of $X_i$ explained by the instrument.
Second stage. The second stage regresses the outcome on the predicted treatment: $Y_i = \alpha + \beta \widehat{X}_i + e_i$ The resulting coefficient estimates the effect of the instrument-induced component of $X_i$ on $Y_i$. The intuition is that $X_i$ contains both endogenous and exogenous variation. The first stage extracts the variation in $X_i$ that comes from $Z_i$. If $Z_i$ is valid, that part of $X_i$ is exogenous. In practice, researchers do not literally run two independent regressions and use ordinary second-stage standard errors. Standard errors must account for the generated regressor $\widehat{X}_i$. Statistical software’s IV or 2SLS commands handle this.
Including controls. Often the model includes controls $W_i$: First stage: $X_i = \pi_0 + \pi_1 Z_i + W_i'\pi_2 + v_i$ Second stage: $Y_i = \alpha + \beta \widehat{X}_i + W_i'\delta + e_i$ The instrument must be relevant conditional on controls: $\operatorname{Cov}(Z_i,X_i \mid W_i) \neq 0$ and valid conditional on controls. Controls can help if they block backdoor paths between the instrument and the outcome. But controls can also hurt if they are bad controls, such as variables affected by the instrument or treatment. As always, the correct controls depend on the causal structure.

10.8 LATE: Local Average Treatment Effect

Instrumental variables often identify a Local Average Treatment Effect, or LATE. This is one of the most important and often misunderstood features of IV. For a binary instrument and binary treatment, units can be grouped according to how their treatment status responds to the instrument. Let: $D_i(1)$ be treatment status if $Z_i=1$, and $D_i(0)$ be treatment status if $Z_i=0$. There are four types of units.

Type	$D_i(0)$	$D_i(1)$	Interpretation
Never-takers	0	0	Do not take treatment regardless of instrument
Always-takers	1	1	Take treatment regardless of instrument
Compliers	0	1	Take treatment only when encouraged or induced by instrument
Defiers	1	0	Do the opposite of the instrument

Under relevance, exclusion, independence, and monotonicity, IV identifies the average treatment effect for compliers: $LATE = \mathbb{E}[Y_i(1)-Y_i(0) \mid D_i(1)>D_i(0)]$ In words:

IV estimates the treatment effect for units whose treatment status was changed by the instrument.

This is not necessarily the ATE for everyone. It is not necessarily the ATT for all treated units. It is the effect for compliers. For example, if distance to college is an instrument for college attendance, the IV estimate applies to people who attend college because they live close enough. It may not apply to people who would attend college regardless of distance or people who would never attend college even if a college were nearby. If a randomized encouragement is used as an instrument for job training participation, the IV estimate applies to people who participate because of the encouragement. LATE can be extremely useful, but it must be interpreted honestly. The correct statement is not:

College increases earnings by $\beta$ for everyone.

A more careful statement is:

Under the IV assumptions, the instrument-induced increase in college attendance raises earnings by $\beta$ for individuals whose college attendance was changed by the instrument.

10.9 Compliance types and policy interpretation

The LATE framework clarifies why IV estimates are often local. Consider a job training program with randomized encouragement. Some workers participate even without encouragement. They are always-takers. Some workers do not participate even with encouragement. They are never-takers. Some workers participate only if encouraged. They are compliers. The IV estimate reflects the effect for compliers. This may be exactly the policy-relevant group if the policy being considered is an encouragement or outreach program. But it may not answer broader questions such as:

What would happen if training became mandatory?
What would happen if training were expanded to all unemployed workers?
What is the effect for highly motivated participants?
What is the effect for people least likely to participate?

The instrument defines the margin of variation. Different instruments can identify different LATEs. For education, a compulsory schooling law may identify the effect of education for students induced to stay in school by the law. A college proximity instrument may identify the effect for students whose attendance depends on distance. A tuition subsidy may identify the effect for students whose attendance depends on price. These may be different populations with different treatment effects. Therefore, IV estimates should be interpreted in relation to the instrument that generated them.

10.10 Weak instruments

A weak instrument is an instrument that has only a weak relationship with the endogenous variable. In the first stage: $X_i = \pi_0 + \pi_1 Z_i + v_i$ weak instruments occur when $\pi_1$ is close to zero. Weak instruments are dangerous because they can produce:

biased IV estimates,
large standard errors,
unstable estimates,
confidence intervals with poor coverage,
estimates highly sensitive to small specification changes,
misleading hypothesis tests.

The intuition is simple. If the instrument barely shifts treatment, then the IV estimate divides by a very small first stage: $\beta_{IV} = \frac{Reduced\ Form}{First\ Stage}$ When the denominator is close to zero, small sampling variation can produce very large changes in the estimate. A common diagnostic in single-endogenous-variable settings is the first-stage $F$-statistic. A traditional rule of thumb is that the first-stage $F$-statistic should exceed 10, although this rule is imperfect and context-dependent. Researchers should report:

the first-stage coefficient,
the first-stage standard error,
the first-stage $F$-statistic,
the magnitude of the first stage,
the economic interpretation of the first stage,
robustness to alternative specifications.

A statistically significant first stage is not enough. The first stage should be strong enough to support reliable inference.

10.11 Invalid instruments

An instrument can fail for many reasons.

Direct effect on the outcome. The instrument may affect the outcome through a channel other than treatment. DAG: $Z \rightarrow X \rightarrow Y$ but also: $Z \rightarrow Y$ Example: distance to college may affect earnings not only through college attendance but also through local labor markets.
Correlation with omitted variables. The instrument may be correlated with unobserved determinants of the outcome. DAG: $A \rightarrow Z$ $A \rightarrow Y$ Example: if families choose neighborhoods partly based on school quality and labor market opportunity, then distance to college may be correlated with family background and local opportunity.
Manipulation of the instrument. The instrument may be chosen or manipulated by individuals. Example: if households move close to colleges because they value education, college distance is not as-good-as-random.
Violation of monotonicity. The instrument may increase treatment for some units but decrease it for others. Example: an encouragement letter might motivate some people to join a program but annoy others and make them less likely to join.
Spillovers and general equilibrium effects. The instrument may affect outcomes for units whose own treatment status does not change. Example: a training program may improve job prospects for participants while reducing job prospects for nonparticipants competing in the same labor market.
Multiple channels. The instrument may bundle several treatments. Example: eligibility for a program may provide job training, counseling, transportation assistance, and employer referrals. If the treatment is defined as training only, the instrument affects the outcome through other channels too. The key lesson is: “A good instrument is not merely a variable that predicts treatment. It must generate treatment variation that is plausibly unrelated to all other determinants of the outcome.”

10.12 Measurement error and IV

Instrumental variables can sometimes address measurement error. Suppose the true model is: $Y_i = \alpha + \beta X_i^* + u_i$ where $X_i^*$ is the true variable of interest. But we observe $X_i$ with measurement error: $X_i = X_i^* + e_i$ If the measurement error $e_i$ is classical, OLS estimates of $\beta$ are biased toward zero. This is called attenuation bias.

An instrument $Z_i$ for $X_i^*$ can help if:

$Z_i$ is correlated with the true variable $X_i^*$,
$Z_i$ is uncorrelated with the measurement error $e_i$,
$Z_i$ affects $Y_i$ only through $X_i^*$.

For example, suppose income is measured with error in survey data. Administrative income records or repeated measures may sometimes be used as instruments for measured survey income, depending on the setting. However, IV does not automatically solve measurement error. The instrument must still satisfy relevance and exclusion.

10.13 IV with multiple instruments and overidentification

Sometimes researchers have more instruments than endogenous variables. For example, suppose $X_i$ is endogenous and there are two instruments, $Z_{1i}$ and $Z_{2i}$. The first stage is: $X_i = \pi_0 + \pi_1 Z_{1i} + \pi_2 Z_{2i} + v_i$ If both instruments are valid, they can improve precision by providing more exogenous variation.

When there are more instruments than endogenous variables, the model is overidentified. Researchers can conduct overidentification tests, such as a Sargan or Hansen test, to examine whether the instruments appear mutually consistent. However, overidentification tests have limits. They do not prove that instruments are valid. They only test whether the instruments produce statistically consistent estimates under certain assumptions. If all instruments are invalid in similar ways, the test may not detect the problem. Overidentification tests should be viewed as diagnostics, not proof of validity. A credible IV design still requires substantive defense of each instrument.

10.14 DAG intuition for instrumental variables

DAGs help clarify IV assumptions. A valid instrument has this structure:

\[Z \rightarrow X \rightarrow Y\]

with possible unobserved confounding between $X$ and $Y$:

\[U \rightarrow X\]

\[U \rightarrow Y\]

The problem is that $X$ and $Y$ are connected both by the causal path:

\[X \rightarrow Y\]

and by the backdoor path:

\[X \leftarrow U \rightarrow Y\]

OLS is biased because of the open backdoor path through $U$. The instrument $Z$ is useful because it shifts $X$ without being connected to $U$.

A valid IV DAG should not have:

\[Z \leftarrow U \rightarrow Y\]

or:

\[Z \rightarrow Y\]

or:

\[Z \rightarrow M \rightarrow Y\]

where $M$ is some non-treatment channel. DAGs also show why controlling for the wrong variables can damage an IV design. For example, controlling for a mediator between $Z$ and $X$ can weaken or eliminate the first stage. Controlling for a collider affected by $Z$ and unobserved determinants of $Y$ can create bias. As always, controls should be chosen using causal reasoning, not mechanically.

10.15 IV versus ordinary regression controls

Regression controls try to solve endogeneity by adjusting for observed confounders. IV tries to solve endogeneity by finding exogenous variation in treatment. Suppose education is endogenous because of ability. A regression-control strategy might estimate: $Wage_i = \alpha + \beta Education_i + \delta AbilityProxy_i + u_i$ This works only if the ability proxy adequately captures the relevant confounding. An IV strategy might use compulsory schooling laws as an instrument: $Education_i = \pi_0 + \pi_1 CompulsoryLaw_i + v_i$ The IV strategy does not require measuring ability perfectly. Instead, it requires that the law-induced variation in education is unrelated to ability and affects wages only through education. Neither approach is automatically better. Each depends on assumptions. Controls are credible when confounders are observed and measured well. IV is credible when a valid source of exogenous treatment variation exists. A weak regression with many controls is not necessarily credible. A clever instrument with an implausible exclusion restriction is not credible either. The quality of the design depends on the quality of the identifying assumptions.

10.16 IV versus randomized controlled trials

An RCT randomly assigns treatment or treatment offer. IV often appears inside RCTs when assignment and actual treatment differ. Suppose individuals are randomly assigned to be offered a program: $Z_i \in \{0,1\}$ where $Z_i=1$ means offered treatment. Actual participation is: $D_i \in \{0,1\}$ Some people offered treatment do not participate. Some people not offered treatment may access treatment elsewhere. The intent-to-treat effect is: $ITT = \mathbb{E}[Y_i \mid Z_i=1] - \mathbb{E}[Y_i \mid Z_i=0]$ This is the effect of being offered treatment. If we want the effect of actually receiving treatment, we may use assignment $Z_i$ as an instrument for participation $D_i$. The IV estimate is: $\frac{\mathbb{E}[Y_i \mid Z_i=1] - \mathbb{E}[Y_i \mid Z_i=0]} {\mathbb{E}[D_i \mid Z_i=1] - \mathbb{E}[D_i \mid Z_i=0]}$ This estimates the treatment effect for compliers: people whose participation status was changed by the offer. This is why RCTs with imperfect compliance often report both:

ITT: effect of assignment or offer,
IV/TOT/LATE: effect of treatment for compliers.

The ITT is often more policy-relevant if the real policy is to offer a program. The IV estimate is more relevant if the question concerns actual participation.

10.17 Common empirical examples

Education and earnings. Question: “What is the causal effect of schooling on earnings?” Problem: Education is endogenous because of ability, family background, motivation, and selection. Possible instruments: Examples/considerations: compulsory schooling laws,; school entry age rules,; distance to college,; quarter of birth in some institutional settings,; tuition changes,; lottery-based school admissions. Main concerns: Examples/considerations: Does the instrument strongly affect schooling?; Does it affect earnings through channels other than schooling?; Who are the compliers?; Is the estimate local to marginal students affected by the instrument?.
Military service and later earnings. Question: “What is the effect of military service on later earnings?” Problem: Military service may be selected. People who serve may differ from those who do not. Possible instrument: Examples/considerations: draft lottery number. Relevance: Lottery number affects probability of military service. Exclusion concern: Lottery number should affect later earnings only through military service. This is more plausible if the lottery is random and does not affect other outcomes except through service-related channels.
Incarceration and labor market outcomes. Question: “What is the effect of incarceration on later employment?” Problem: People who are incarcerated differ from those who are not in many unobserved ways. Possible instrument: Examples/considerations: random assignment to stricter versus more lenient judges. Relevance: Judge assignment affects probability or length of incarceration. Exclusion concern: Judge assignment should affect later employment only through incarceration, not through other sentencing conditions, court experiences, fines, probation terms, or stigma channels independent of incarceration.
Health insurance and healthcare use. Question: “What is the effect of insurance coverage on healthcare use?” Problem: People with insurance differ from people without insurance. Possible instruments: Examples/considerations: eligibility thresholds,; policy expansions,; randomized insurance lotteries,; employer mandates. Concerns: Eligibility may affect outcomes through income, labor supply, or program interactions. Policy changes may coincide with other changes.

10.18 What to report in an IV study.

A credible IV analysis should report more than the second-stage coefficient. At minimum, researchers should report:

the causal question,
the endogenous variable,
the instrument,
the first-stage relationship,
the reduced-form relationship,
the IV estimate,
the first-stage strength,
the identifying assumptions,
evidence supporting relevance,
evidence supporting exclusion,
who the compliers likely are,
whether the estimate is LATE, ATT, ATE, or another estimand,
robustness checks,
sensitivity to alternative controls,
standard errors appropriate to the design,
external validity limitations.

The reduced form should not be hidden. It is often the most transparent estimate because it shows the effect of the instrument itself on the outcome. The first stage should also be interpreted substantively, not merely statistically. For example, saying that an instrument increases schooling by 0.03 years may suggest a very small first stage, even if statistically significant in a large sample.

10.19 Common mistakes with IV.

Mistake	Why it matters
Treating relevance as validity	A strong first stage does not make an instrument valid; the instrument can strongly predict treatment and still violate exclusion.
Treating exclusion as testable	The exclusion restriction is generally not directly testable and must be defended with theory, institutional knowledge, and indirect evidence.
Forgetting LATE	IV often estimates the effect for compliers, not the average effect for everyone. Researchers should identify who the instrument actually moves.
Using weak instruments	Weak instruments can generate unstable, biased, and misleading estimates even when the first stage is statistically significant.
Controlling for post-instrument variables	Controls affected by the instrument can block causal pathways or introduce bias; controls should be chosen from the causal graph.
Ignoring the reduced form	If the instrument has no clear effect on the outcome, the IV estimate may be driven by a small denominator or fragile specification.
Overgeneralizing the estimate	Different instruments identify different margins; compulsory schooling laws, tuition subsidies, and distance-to-college instruments may apply to different groups.
Using too many instruments carelessly	Many instruments can increase finite-sample bias and make overidentification tests misleading.

10.20 Application checklist for IV.

Dimension	Questions to answer
Causal question	What is the treatment or endogenous variable $X$? What is the outcome $Y$? What effect is being estimated?
Why OLS is not credible	Is the problem omitted variables, reverse causality, simultaneity, selection, measurement error, or another source of endogeneity?
Instrument	What variable $Z$ shifts $X$, and what institutional or behavioral mechanism links $Z$ to $X$?
Relevance	Does $Z$ strongly and meaningfully predict $X$? What is the first-stage estimate and strength?
Exclusion	Could $Z$ affect $Y$ through channels other than $X$? Are direct effects, spillovers, or omitted variables plausible?
Independence	Is $Z$ as-good-as-random? Could units manipulate or select into $Z$? Are baseline covariates balanced?
Monotonicity	Does the instrument move treatment in the same direction for all relevant units, or could there be defiers?
Compliers	Whose treatment status changes because of the instrument, and are they policy-relevant?
Reduced form	Does the instrument affect the outcome in a way that is substantively meaningful and consistent with the mechanism?
Controls	Are controls pre-instrument and pre-treatment? Could any controls be mediators or colliders?
Inference	Are standard errors appropriate? Are instruments weak? Are errors clustered or serially correlated?
Interpretation	Is the estimate ATE, ATT, LATE, or something else? For whom does it apply, under what assumptions, and with what threats?

10.21 Summary

Instrumental variables are used when the treatment or explanatory variable is endogenous. The basic structural problem is: $Y_i = \alpha + \beta X_i + u_i$ with $\operatorname{Cov}(X_i,u_i) \neq 0$. An instrument $Z_i$ helps if it shifts $X_i$ while being otherwise unrelated to the unobserved determinants of $Y_i$. The two core IV assumptions are relevance and exclusion. Relevance requires: $\operatorname{Cov}(Z_i,X_i) \neq 0$ Exclusion requires that $Z_i$ affects $Y_i$ only through $X_i$. Two-stage least squares estimates the effect of the instrument-induced component of $X_i$ on $Y_i$. The first stage is: $X_i = \pi_0 + \pi_1 Z_i + v_i$ The second stage is: $Y_i = \alpha + \beta \widehat{X}_i + e_i$ With binary instruments and treatments, IV often identifies a Local Average Treatment Effect: $LATE = \mathbb{E}[Y_i(1)-Y_i(0) \mid D_i(1)>D_i(0)]$ This is the effect for compliers: units whose treatment status is changed by the instrument. The power of IV is that it can isolate causal variation when ordinary treatment variation is endogenous. The danger of IV is that validity depends on strong assumptions, especially the exclusion restriction. The central question in any IV design is:

Does the instrument create variation in treatment that is both strong enough and clean enough to support a causal interpretation?

A credible IV study must explain the source of variation, show the first stage, defend exclusion, identify the complier population, address weak instruments, and interpret the estimate locally and carefully.

11. Difference-in-Differences

11.1 Why Difference-in-Differences matters

Difference-in-Differences, often abbreviated as DiD, is one of the most important quasi-experimental methods in empirical economics. It is used when some units receive a treatment, policy, or shock while other comparable units do not, and we observe both groups before and after treatment. The core idea is simple:

Compare how outcomes change over time for a treated group to how outcomes change over time for a control group.

DiD is useful because treated and control groups may differ in levels before treatment. The treated group may start richer, poorer, healthier, less employed, more educated, more urban, or more productive than the control group. A simple post-treatment comparison may therefore be misleading. DiD removes two kinds of differences:

Permanent differences between treated and control groups, such as stable differences in geography, institutions, culture, baseline productivity, or demographics. 2. Common time shocks affecting both groups, such as national recessions, inflation, federal policy changes, technological trends, or macroeconomic cycles. What remains is the difference in changes. For example, suppose one state raises its minimum wage and another state does not. Employment may be lower in the treated state even before the policy. A post-policy comparison of employment levels would confuse the policy effect with pre-existing differences. DiD instead asks:

Did employment change more in the treated state than in the control state after the policy?

This makes DiD a natural design for studying policy changes, institutional reforms, staggered rollouts, legal changes, and shocks that affect some units but not others. Common applications include estimating the effects of:

minimum wage laws,
tax changes,
unemployment insurance expansions,
school reforms,
healthcare expansions,
policing interventions,
environmental regulations,
labor market policies,
trade shocks,
voting laws,
welfare reforms,
infrastructure programs.

The power of DiD comes from its counterfactual logic. The danger of DiD comes from the fact that this logic depends on a strong assumption: parallel trends.

11.2 The basic two-group, two-period setup

The simplest DiD design has two groups and two time periods. There is:

one treated group,
one control group,
one pre-treatment period,
one post-treatment period.

Let $G_i$ indicate whether unit $i$ belongs to the treated group: $G_i = 1$ if unit $i$ is in the treated group, and $G_i = 0$ if unit $i$ is in the control group. Let $Post_t$ indicate whether time period $t$ is after treatment: $Post_t = 1$ in the post-treatment period, and $Post_t = 0$ in the pre-treatment period. Treatment exposure occurs only for the treated group in the post-treatment period: $D_{it} = G_i \times Post_t$ The treated group before treatment has $G_i=1$ and $Post_t=0$, so: $D_{it}=0$ The treated group after treatment has $G_i=1$ and $Post_t=1$, so: $D_{it}=1$ The control group is never treated, so $D_{it}=0$ in both periods. The observed data can be summarized in a two-by-two table:

Group	Before	After	Change
Treated	$\bar{Y}_{T,Before}$	$\bar{Y}_{T,After}$	$\bar{Y}_{T,After}-\bar{Y}_{T,Before}$
Control	$\bar{Y}_{C,Before}$	$\bar{Y}_{C,After}$	$\bar{Y}_{C,After}-\bar{Y}_{C,Before}$

The DiD estimator is: $\widehat{\tau}_{DiD} = (\bar{Y}_{T,After}-\bar{Y}_{T,Before}) - (\bar{Y}_{C,After}-\bar{Y}_{C,Before})$ This is the change in the treated group minus the change in the control group. That is why the method is called Difference-in-Differences.

11.3 Numerical example

Suppose we want to study the effect of a minimum wage increase on employment. One state raises the minimum wage. Another similar state does not.

Group	Before	After	Change
Treated state	100	95	$95-100=-5$
Control state	100	98	$98-100=-2$

The treated state’s employment fell by 5 units. The control state’s employment fell by 2 units. The DiD estimate is: $(-5)-(-2)=-3$ The interpretation is:

Employment in the treated state fell by 3 more units than it would have if it had followed the same trend as the control state.

The phrase if it had followed the same trend as the control state is crucial. DiD does not simply compare treated before to treated after. It uses the control group to estimate the counterfactual trend for the treated group. The before-after estimate for the treated group alone is: $95-100=-5$ But this may overstate the policy effect if employment would have fallen anyway because of a recession or seasonal downturn. The control group fell by 2 units, suggesting that some of the decline may have occurred even without the policy. DiD subtracts that common decline.

11.4 The counterfactual logic of DiD

DiD is a method for constructing a missing counterfactual. For the treated group after treatment, we observe the treated potential outcome: $Y_{it}(1)$ But we do not observe what the treated group would have experienced after treatment in the absence of treatment: $Y_{it}(0)$ The missing object is: $\mathbb{E}[Y_{it}(0) \mid G_i=1, Post_t=1]$ This means:

The average outcome the treated group would have had in the post-treatment period if it had not been treated.

DiD estimates this missing counterfactual by taking the treated group’s pre-treatment outcome and adding the control group’s observed change over time: $\widehat{\mathbb{E}}[Y(0) \mid G=1, Post=1] = \bar{Y}_{T,Before} + (\bar{Y}_{C,After}-\bar{Y}_{C,Before})$ Then the DiD estimate is: $\widehat{\tau}_{DiD} = \bar{Y}_{T,After} - \left[\bar{Y}_{T,Before}+ (\bar{Y}_{C,After}-\bar{Y}_{C,Before})\right]$ Rearranging gives the standard formula: $\widehat{\tau}_{DiD} = (\bar{Y}_{T,After}-\bar{Y}_{T,Before}) - (\bar{Y}_{C,After}-\bar{Y}_{C,Before})$ So DiD asks:

How far is the treated group’s post-treatment outcome from where it would have been if it had changed like the control group?

That is the essence of the design.

11.5 The parallel trends assumption

The core identifying assumption in DiD is the parallel trends assumption. In the two-group, two-period case, parallel trends says: $\mathbb{E}[Y_{After}(0)-Y_{Before}(0) \mid G=1] = \mathbb{E}[Y_{After}(0)-Y_{Before}(0) \mid G=0]$ In words:

In the absence of treatment, the treated group and the control group would have experienced the same average change in outcomes.

This assumption is about untreated potential outcomes. It is not directly testable because we do not observe the treated group’s untreated post-treatment outcome. Parallel trends does not require treated and control groups to have the same outcome levels. The treated group may always have higher or lower outcomes than the control group. What matters is whether the groups would have changed similarly absent treatment. For example, suppose treated cities have higher crime than control cities before a policing reform. That does not automatically invalidate DiD. DiD can still be credible if, absent the reform, crime in treated and control cities would have followed similar trends. The key comparison is trends, not levels. A useful way to think about the assumption is:

The control group tells us how the treated group would have changed over time if treatment had not occurred.

If that statement is credible, DiD may be credible. If that statement is not credible, DiD may be biased.

11.6 Why DiD can work even when groups differ in levels

Suppose the treated group has a permanently higher outcome than the control group. For example, treated cities may have higher baseline crime than control cities because they are larger and more urban. Let the untreated outcome be: $Y_{it}(0)=\alpha_i+\lambda_t+u_{it}$ where:

$\alpha_i$ is a unit-specific permanent component,
$\lambda_t$ is a time-specific shock common to all units,
$u_{it}$ is an idiosyncratic shock.

If treated and control units differ in $\alpha_i$, they differ in levels. But if $\alpha_i$ is constant over time, differencing removes it. The treated group’s change is: $Y_{i,After}(0)-Y_{i,Before}(0) = (\lambda_{After}-\lambda_{Before})+(u_{i,After}-u_{i,Before})$ The permanent component $\alpha_i$ drops out. Similarly, if $\lambda_t$ is a common time shock affecting both groups, subtracting the control group’s change removes it. This is why DiD is powerful. It can handle permanent group differences and common time shocks. But it cannot handle untreated trends that differ systematically between treated and control groups. If the treated group would have been rising faster or falling faster even without treatment, then DiD will attribute that differential trend to the treatment.

11.7 Regression form of the basic DiD model

The two-group, two-period DiD estimator can be written as a regression: $Y_{it}=\alpha+\delta G_i+\lambda Post_t+\beta(G_i \times Post_t)+u_{it}$ where:

$G_i$ indicates membership in the treated group,
$Post_t$ indicates the post-treatment period,
$G_i \times Post_t$ indicates treatment exposure,
$\beta$ is the DiD estimate.

The coefficient $\delta$ captures baseline differences between treated and control groups. The coefficient $\lambda$ captures common changes over time. The coefficient $\beta$ captures the extra post-treatment change in the treated group relative to the control group. In the simple two-by-two case: $\hat{\beta}=\widehat{\tau}_{DiD}$ The regression form is useful because it allows researchers to add covariates, fixed effects, clustered standard errors, multiple time periods, and multiple groups.

11.8 DiD with unit and time fixed effects

With panel data, the common DiD specification is: $Y_{it}=\alpha_i+\lambda_t+\beta D_{it}+u_{it}$ where:

$\alpha_i$ are unit fixed effects,
$\lambda_t$ are time fixed effects,
$D_{it}$ is treatment status,
$\beta$ is the DiD coefficient.

Unit fixed effects control for all time-invariant differences across units. Time fixed effects control for shocks common to all units in a given period. The treatment coefficient $\beta$ is identified by within-unit changes in treatment status relative to changes among units not treated at that time. For example, if some states raise their minimum wage and others do not, $\beta$ compares employment changes in states that raise the minimum wage to employment changes in states that do not, after controlling for permanent state differences and common year shocks. This specification is often called a two-way fixed effects model because it includes both unit and time fixed effects. The model is: $Y_{it}=\alpha_i+\lambda_t+\beta D_{it}+u_{it}$ The identifying assumption is not merely that the regression includes fixed effects. The identifying assumption is still a version of parallel trends:

In the absence of treatment, treated and comparison units would have followed similar trends after accounting for unit and time fixed effects.

Fixed effects do not guarantee parallel trends. They only remove certain types of confounding.

11.9 What fixed effects remove and what they do not remove

Unit fixed effects remove time-invariant confounders. Examples include:

geography,
historical institutions,
baseline culture,
permanent productivity differences,
stable school quality,
long-run demographic composition,
persistent regional characteristics.

Time fixed effects remove shocks common to all units in a given period. Examples include:

national recessions,
federal policy changes,
inflation,
nationwide technological changes,
common seasonal shocks,
national public health shocks.

But DiD with fixed effects does not automatically remove time-varying confounders. Examples include:

a treated state raising the minimum wage during a local recession,
a city increasing policing exactly when crime is already rising,
a school adopting tutoring when student performance is declining,
a hospital adopting a reform when patient severity is changing,
a country changing trade policy during a political crisis.

If time-varying shocks affect treatment adoption and outcomes, then $\mathbb{E}[u_{it} \mid D_{it}] \neq 0$ and the DiD estimate may be biased. The central question remains:

Is treatment timing plausibly unrelated to untreated outcome trends?

11.10 DiD with covariates

Researchers often estimate DiD models with additional covariates: $Y_{it}=\alpha_i+\lambda_t+\beta D_{it}+X_{it}'\gamma+u_{it}$ where $X_{it}$ is a vector of observed controls. Covariates can improve precision and adjust for observed time-varying differences, but they must be chosen carefully. Good controls are usually pre-treatment variables or time-varying variables that are not themselves affected by treatment. Bad controls include:

mediators,
post-treatment outcomes,
variables affected by treatment,
colliders,
variables that proxy for treatment effects.

For example, suppose a job training program affects employment, and employment affects later earnings: $Training \rightarrow Employment \rightarrow Earnings$ If the outcome is earnings and we control for employment after training, we may block part of the treatment effect. Similarly, if a minimum wage increase affects firm closures, and we control for the number of surviving firms, we may condition on a post-treatment variable and distort the effect. Covariates do not rescue a DiD design if parallel trends is implausible. They can help only if they address specific, credible sources of imbalance without creating post-treatment bias.

11.11 Visualizing DiD

A good DiD analysis usually begins with a graph. The graph should show average outcomes over time for treated and control groups. A credible DiD graph often has this pattern:

Treated and control groups may have different levels.
Before treatment, the groups move in roughly parallel trends.
Around treatment, the treated group changes relative to the control group.
After treatment, the gap evolves in a way consistent with the hypothesized effect.

The graph helps assess whether the identifying assumption is plausible. However, graphs are not proof. Parallel pre-trends support the design, but they do not guarantee parallel post-treatment counterfactual trends. A graph can reveal obvious problems, such as:

treated and control groups trending differently before treatment,
outcome changes beginning before treatment,
large shocks coinciding with treatment,
unstable or noisy pre-period data,
control group discontinuities unrelated to treatment.

Visual evidence should be combined with institutional knowledge, statistical tests, placebo exercises, robustness checks, and sensitivity analysis.

11.12 Event studies

An event study extends DiD by estimating treatment effects at multiple times before and after treatment. Instead of estimating one average post-treatment effect, an event study estimates dynamic effects relative to the treatment date. A common event-study specification is: $Y_{it}=\alpha_i+\lambda_t+ \sum_{k \neq -1}\beta_k \mathbf{1}\{t-T_i=k\}+u_{it}$ where:

$T_i$ is the treatment date for unit $i$,
$k$ indexes time relative to treatment,
$k=-1$ is often omitted as the reference period,
$\beta_k$ measures the difference between treated and comparison units $k$ periods from treatment.

The coefficients for $k<0$ are called leads. They estimate pre-treatment differences relative to the reference period. The coefficients for $k>0$ are called lags. They estimate post-treatment effects. Event studies serve two major purposes. First, they help evaluate pre-trends. If the design is credible, pre-treatment coefficients should generally be close to zero: $\beta_k \approx 0 \quad \text{for } k<0$ Large pre-treatment effects suggest that treated and comparison units were already evolving differently before treatment. Second, event studies show dynamics. Treatment effects may appear immediately, grow over time, fade out, or emerge only after a delay. For example, a schooling reform may not affect earnings until students enter the labor market years later. A tax change may affect firm behavior quickly. A health intervention may have both short-run and long-run effects. Event studies are useful because they make the timing of effects visible.

11.13 Interpreting pre-trend tests

Pre-trend tests are common in DiD, but they require careful interpretation. A typical test asks whether the pre-treatment event-study coefficients are jointly equal to zero: $H_0: \beta_k=0 \quad \text{for all } k<0$ Failing to reject this null does not prove parallel trends. It may simply mean the test has low power. For example, if the sample is small or outcomes are noisy, large violations of parallel trends may be difficult to detect statistically. Rejecting the null is stronger evidence of a problem, but even then researchers should examine the magnitude and pattern of the pre-trends, not only the p-value. Important questions include:

Are pre-treatment differences economically meaningful?
Are they trending in the same direction as the estimated treatment effect?
Do effects appear before treatment, suggesting anticipation or endogenous timing?
Are results sensitive to including unit-specific trends?
Are pre-trends visible graphically?

Pre-trend tests are diagnostic tools. They are not automatic validators.

11.14 Anticipation effects

DiD can fail if units change behavior before treatment officially begins. Anticipation effects occur when people, firms, or governments respond to expected treatment before it is implemented. Examples:

Firms reduce hiring before a minimum wage increase takes effect.
Consumers buy goods before a tax increase begins.
Workers retire before pension rules change.
Schools adjust behavior before accountability rules are enforced.
Polluters change emissions before environmental regulations begin.

In such cases, the pre-treatment period may already be affected by treatment expectations. This violates the idea that pre-treatment outcomes represent untreated potential outcomes. Researchers may address anticipation by:

redefining treatment timing to the announcement date,
excluding periods between announcement and implementation,
estimating event-study leads,
using institutional details to understand when agents learned about the policy,
conducting sensitivity checks with alternative treatment dates.

The key question is:

When did the treatment begin affecting behavior?

The answer may differ from the formal implementation date.

11.15 Spillovers and contamination

DiD assumes that the control group represents what would have happened to the treated group without treatment. This can fail if treatment affects the control group. Spillovers occur when treatment of one unit affects outcomes for other units. Examples:

A policing intervention in one neighborhood displaces crime to nearby neighborhoods.
A job training program helps participants compete for jobs, reducing job opportunities for nonparticipants.
A minimum wage increase in one city affects neighboring labor markets.
A school reform changes peer behavior across schools.
A vaccination campaign affects disease risk for untreated people.

When spillovers affect the control group, the estimated DiD effect may be too small, too large, or difficult to interpret. Suppose treatment improves outcomes for treated units but also improves outcomes for control units through spillovers. Then the treated-control difference may understate the total effect. Suppose treatment benefits treated workers by displacing untreated workers. Then the treated-control difference may overstate social gains. Researchers should ask:

Are control units geographically close to treated units?
Do markets connect treated and control units?
Could treatment affect prices, wages, peers, competitors, or behavior elsewhere?
Is the estimated effect partial equilibrium or general equilibrium?
Does the estimand include or exclude spillover effects?

Spillovers are not merely a nuisance. They often change the policy interpretation.

11.16 Composition changes

DiD can also fail if the composition of treated or control groups changes over time in ways related to treatment. For example:

A minimum wage increase may cause some firms to exit the sample.
A school reform may cause students to move schools.
A health policy may change which patients seek care.
A labor market shock may cause selective migration.
A survey panel may lose respondents differently across groups.

If the units observed after treatment are not comparable to the units observed before treatment, outcome changes may reflect compositional shifts rather than treatment effects. Suppose a job training program causes low-earning workers to leave the sample because they find informal work not measured in administrative data. Measured average earnings among remaining workers may change for reasons unrelated to true earnings effects. Researchers should examine:

sample sizes over time,
attrition rates,
migration patterns,
entry and exit of firms,
changes in observed covariates,
whether treatment affects measurement or reporting.

A DiD design is strongest when the population being compared is stable or when changes in composition can be measured and addressed.

11.17 Treatment timing and endogenous adoption

Policies are often not adopted randomly. Governments, firms, schools, and individuals choose when to adopt treatments. Treatment timing is endogenous if units adopt treatment in response to expected or current outcome trends. Examples:

Cities increase policing when crime is rising.
Schools adopt tutoring when test scores are falling.
States raise minimum wages when labor markets are strong.
Firms adopt technology when demand is increasing.
Hospitals adopt quality reforms after poor performance.

Endogenous adoption threatens parallel trends because treated units may have different untreated trends precisely because of why they adopted treatment. For example, if cities increase policing when crime is already rising, a DiD estimate may incorrectly attribute rising crime to policing. Institutional knowledge is essential. Researchers should ask:

Why did treatment occur when it did?
Were adoption decisions related to outcome trends?
Did policymakers respond to local shocks?
Were treated units selected because of high need or high capacity?
Were control units truly comparable?

A credible DiD design often requires a policy or shock whose timing is plausibly unrelated to untreated outcome trends.

11.18 Staggered adoption

Many real-world DiD settings involve staggered adoption. Different units adopt treatment at different times. For example:

states adopt laws in different years,
countries join trade agreements at different times,
schools implement reforms in waves,
hospitals adopt technology gradually,
cities roll out infrastructure programs over time.

A common two-way fixed effects model is: $Y_{it}=\alpha_i+\lambda_t+\beta D_{it}+u_{it}$ where $D_{it}=1$ after unit $i$ adopts treatment. For a long time, researchers interpreted $\hat{\beta}$ as a weighted average treatment effect. However, staggered adoption creates complications when treatment effects are heterogeneous across groups or over time. The problem is that already-treated units can serve as controls for later-treated units. If treatment effects evolve over time, these comparisons can be misleading. For example, suppose early-treated states experience growing effects over time. When later-treated states adopt, the model may compare them to already-treated states whose outcomes have already changed because of treatment. This can contaminate the estimate. Traditional two-way fixed effects DiD can produce weights that are hard to interpret and, in some cases, negative. The practical lesson is:

In staggered adoption settings, a single two-way fixed effects coefficient may not equal a clean average of meaningful treatment effects.

Modern DiD methods often estimate group-time average treatment effects and compare treated units only to valid comparison groups, such as never-treated or not-yet-treated units.

11.19 Group-time average treatment effects

In staggered adoption designs, it is useful to define treatment effects by group and time. Let $g$ denote the period when a group first receives treatment. Let $t$ denote calendar time. A group-time average treatment effect can be written as: $ATT(g,t)=\mathbb{E}[Y_t(1)-Y_t(0) \mid G_i=g]$ for periods $t \geq g$. This asks:

What is the average treatment effect at time $t$ for units first treated in period $g$?

This approach recognizes that treatment effects may differ by cohort and by time since treatment. Effects may vary because:

early adopters differ from late adopters,
implementation quality differs across cohorts,
effects grow or fade over time,
macroeconomic conditions differ across adoption periods,
treatment intensity changes across groups.

After estimating $ATT(g,t)$, researchers can aggregate effects in transparent ways:

by event time,
by treatment cohort,
across calendar periods,
into an overall average treatment effect on treated units.

The advantage is interpretability. Rather than relying on a single opaque two-way fixed effects coefficient, the researcher can show which comparisons identify which effects.

11.20 Never-treated and not-yet-treated controls

In staggered DiD, comparison groups matter. A never-treated control group consists of units that never receive treatment during the sample period. A not-yet-treated control group consists of units that have not yet received treatment by a given time, even if they receive treatment later. Using not-yet-treated controls can be reasonable if future treatment timing is not driven by current untreated outcome trends. However, if future-treated units are already on different trajectories before treatment, they may not be valid controls. For example, states that will adopt a minimum wage increase next year may already have strengthening labor markets this year. Using them as controls for earlier adopters may create bias. The choice of comparison group should be justified substantively, not mechanically. Researchers should ask:

Are never-treated units comparable to treated units?
Are not-yet-treated units valid controls before they adopt?
Do future-treated units show different pre-trends?
Are adoption cohorts systematically different?
Does the design rely on comparisons with already-treated units?

Good DiD practice makes the comparison group explicit.

11.21 Unit-specific trends

Sometimes researchers include unit-specific linear trends: $Y_{it}=\alpha_i+\lambda_t+\theta_i t+\beta D_{it}+u_{it}$ where $\theta_i t$ allows each unit to have its own linear trend. This can address the concern that treated and control units were trending differently before treatment. However, unit-specific trends must be used carefully. They can help if untreated trends are approximately linear and differ across units. But they can also create problems:

They may absorb part of the treatment effect if effects evolve gradually.
They may extrapolate trends beyond what the data support.
They may make estimates sensitive to functional form.
They may hide meaningful pre-trend violations rather than solve them.

For example, if treatment effects grow gradually after adoption, a unit-specific trend may treat part of that growth as continuation of the pre-treatment trend rather than as treatment effect. Unit-specific trends are not a substitute for a credible design. They are a modeling choice that should be justified and tested.

11.22 Standard errors and clustering

DiD designs often use panel data, where observations are correlated within units over time. For example, state-level employment outcomes are likely serially correlated. A shock to employment in one year may persist into later years. If standard errors ignore this correlation, they may be too small, leading to false statistical significance. A common practice is to cluster standard errors at the level of treatment assignment. If treatment varies at the state level, standard errors should usually be clustered by state. If treatment varies at the school level, cluster by school. If treatment varies at the firm level, cluster by firm. The idea is that treatment assignment creates correlated errors within treated units over time. With few clusters, conventional clustered standard errors may still be unreliable. Researchers may need methods such as:

wild cluster bootstrap,
randomization inference,
aggregation to the treatment level,
careful sensitivity analysis,
conservative inference procedures.

Correct inference is not a technical afterthought. In DiD, serial correlation and clustering can strongly affect conclusions.

11.23 DiD with repeated cross-sections

DiD does not always require observing the same units over time. Sometimes researchers use repeated cross-sections. For example, a labor force survey may sample different individuals each year, but the researcher observes people in treated and control states before and after a policy. A repeated cross-section DiD might estimate: $Y_{ist}=\alpha_s+\lambda_t+\beta D_{st}+X_{ist}'\gamma+u_{ist}$ where:

$i$ indexes individuals,
$s$ indexes states,
$t$ indexes time,
$D_{st}$ is state-level treatment,
$X_{ist}$ are individual covariates.

The design compares average outcomes for comparable populations in treated and control states over time. The key concern is composition. If the types of individuals sampled in each state change over time, estimated effects may partly reflect changing sample composition. Researchers may address this by:

controlling for demographic covariates,
reweighting samples,
restricting to stable populations,
checking covariate balance over time,
using administrative data when possible.

The parallel trends assumption still concerns untreated potential outcomes for the relevant populations.

11.24 Difference-in-difference-in-differences

Difference-in-difference-in-differences, often called triple differences or DDD, adds another comparison dimension. Suppose a policy affects one group within treated states but not another group. For example, a minimum wage increase may affect low-wage workers more than high-wage workers. A DDD design compares:

treated versus control states,
before versus after the policy,
affected versus less-affected workers.

The idea is to remove additional confounding trends. A simple DDD estimand is: $DDD = \left[(\Delta Y_{T,Affected}-\Delta Y_{C,Affected}) - (\Delta Y_{T,Unaffected}-\Delta Y_{C,Unaffected})\right]$ This asks whether the treated-control DiD is larger for the affected group than for the unaffected group. DDD can be useful when treated and control areas have different trends, but those trends affect both affected and unaffected groups similarly. However, DDD has its own assumption:

In the absence of treatment, the difference between affected and unaffected groups would have evolved similarly in treated and control units.

Adding another difference does not eliminate the need for assumptions. It changes the identifying assumption.

11.25 DiD and DAG intuition

A simple DiD design can be represented using causal logic. Suppose treatment is adopted by some units after a policy date:

\[G_i \rightarrow D_{it}\]

and treatment affects the outcome:

\[D_{it} \rightarrow Y_{it}\]

There may also be unit-level confounders:

\[A_i \rightarrow G_i\]

\[A_i \rightarrow Y_{it}\]

If $A_i$ is time-invariant, unit fixed effects can remove it. There may also be common time shocks:

\[\lambda_t \rightarrow Y_{it}\]

Time fixed effects can remove shocks common to all units. But DiD fails if there are time-varying confounders that affect both treatment and outcomes:

\[W_{it} \rightarrow D_{it}\]

\[W_{it} \rightarrow Y_{it}\]

For example, if rising crime causes a city to adopt policing reform and also affects future crime, then $W_{it}$ creates a time-varying backdoor path. DiD is credible when the remaining untreated outcome trends are comparable after accounting for fixed group differences and common time shocks.

11.26 Example: minimum wage and employment

Suppose State A raises its minimum wage in 2025. State B does not. The outcome is restaurant employment. A naive before-after comparison in State A asks: $\bar{Y}_{A,2026}-\bar{Y}_{A,2024}$ But this may capture many other changes between 2024 and 2026. A DiD design estimates: $(\bar{Y}_{A,2026}-\bar{Y}_{A,2024}) - (\bar{Y}_{B,2026}-\bar{Y}_{B,2024})$ The control state adjusts for changes that would have happened even without the minimum wage increase. The identifying assumption is:

In the absence of the minimum wage increase, restaurant employment in State A would have changed like restaurant employment in State B.

Threats include:

State A and State B had different pre-treatment employment trends.
State A adopted the policy because its labor market was unusually strong.
Another policy changed in State A at the same time.
Firms anticipated the policy and adjusted before 2025.
Workers or firms moved across state borders.
The control state was affected by spillovers.

A strong study would show pre-treatment trends, justify the control group, check robustness to alternative controls, examine anticipation, and cluster standard errors at the state level.

11.27 Example: healthcare expansion and insurance coverage

Suppose some states expand a public health insurance program while others do not. The outcome is insurance coverage. A DiD model might be: $Coverage_{st}=\alpha_s+\lambda_t+\beta Expansion_{st}+u_{st}$ where:

$Coverage_{st}$ is the insurance coverage rate in state $s$ and year $t$,
$Expansion_{st}$ equals 1 after state $s$ expands the program,
$\alpha_s$ are state fixed effects,
$\lambda_t$ are year fixed effects.

The coefficient $\beta$ estimates the change in coverage in expansion states relative to non-expansion states. The identifying assumption is:

Without expansion, coverage in expansion states would have followed the same trend as coverage in non-expansion states.

This may be plausible if pre-expansion trends were similar and expansion timing was not driven by differential coverage trends. Threats include:

expansion states were already trending differently,
state economic conditions changed differently,
outreach campaigns differed across states,
other healthcare policies changed at the same time,
migration changed the composition of state populations.

The study should examine pre-trends and possibly estimate event-study effects around expansion.

11.28 Example: education reform and test scores

Suppose a school district introduces a new curriculum in some schools but not others. The outcome is student test scores. A DiD design compares test score changes in reform schools to test score changes in non-reform schools. A possible model is: $Score_{ist}=\alpha_s+\lambda_t+\beta Reform_{st}+X_{ist}'\gamma+u_{ist}$ where:

$i$ indexes students,
$s$ indexes schools,
$t$ indexes years,
$Reform_{st}$ indicates whether school $s$ has implemented the reform,
$X_{ist}$ includes student characteristics.

The key assumption is:

In the absence of the reform, test scores in reform schools would have followed trends similar to test scores in comparison schools.

Possible threats:

schools adopted the reform because scores were falling,
reform schools had different student composition changes,
teachers changed schools in response to the reform,
students transferred into or out of reform schools,
other school policies changed at the same time.

This example shows why DiD must consider selection into treatment and composition changes.

11.29 Common mistakes in DiD.

Treating DiD as automatically causal. DiD is not automatically causal. It depends on parallel trends and other assumptions. Including unit and time fixed effects does not guarantee identification.
Comparing levels instead of trends. DiD does not require treated and control groups to have equal levels. It requires comparable untreated trends. Rejecting a control group simply because levels differ can be a mistake. Accepting a control group simply because levels are similar can also be a mistake.
Ignoring pre-trends. If treated and control groups were already moving differently before treatment, the DiD estimate may be biased. Pre-trends should be shown graphically when possible.
Ignoring anticipation. Treatment may affect behavior before formal implementation. Researchers should consider announcement dates and expectation effects.
Using bad controls. Controlling for variables affected by treatment can bias the estimate. Covariates should be selected based on causal reasoning, not simply predictive power.
Ignoring staggered adoption problems. In staggered adoption settings, traditional two-way fixed effects can be hard to interpret when effects are heterogeneous. Researchers should examine treatment timing, cohort-specific effects, and valid comparison groups.
Clustering at the wrong level. Standard errors should usually be clustered at the level of treatment assignment. Failure to account for serial correlation can produce misleading statistical significance.
Ignoring spillovers. If treatment affects control units, the comparison group may be contaminated. This is especially important in spatial, labor market, peer, and general equilibrium settings.

11.30 Application checklist.

When evaluating or conducting a DiD study, use the following checklist.

Define treatment. What is the policy, intervention, or shock? When does it begin? Is treatment binary, continuous, staggered, or varying in intensity?
Define treated and control groups. Which units are treated? Which units serve as controls? Are controls never treated, not-yet treated, or less exposed?
Define timing. What is the pre-treatment period? What is the post-treatment period? Was there an announcement period? Could anticipation occur?
Define the estimand. Is the goal to estimate ATT? A dynamic effect? A group-time effect? A short-run or long-run effect?
State the parallel trends assumption. What exactly must be true for the control group to represent the treated group’s counterfactual trend? Write the assumption in words.
Examine pre-treatment trends. Do treated and control groups move similarly before treatment? Are pre-trend differences large or small? Are results supported by graphs and event studies?
Investigate treatment timing. Why did treatment happen when it did? Was adoption related to outcome trends? Were policymakers responding to shocks?
Check for simultaneous changes. Did another policy, shock, or institutional change occur at the same time? Could it explain the estimated effect?
Check for composition changes. Did the sample change after treatment? Was there migration, attrition, entry, exit, or selection?
Check for spillovers. Could treatment affect control units? Are treated and control units connected through markets, geography, peers, or institutions?
Choose controls carefully. Are controls pre-treatment or plausibly unaffected by treatment? Could any controls be mediators, colliders, or post-treatment variables?
Use appropriate inference. At what level is treatment assigned? Are standard errors clustered at that level? Are there enough clusters?
Conduct robustness checks. Try alternative control groups, time windows, specifications, and outcome definitions. Use placebo treatments or placebo outcomes when possible.
Interpret the result carefully. What comparison identifies the estimate? For whom is the effect estimated? Does it generalize beyond the sample and time period?

11.31 Summary

Difference-in-Differences estimates causal effects by comparing changes over time in treated and control groups. The basic estimator is: $\widehat{\tau}_{DiD} = (\bar{Y}_{T,After}-\bar{Y}_{T,Before}) - (\bar{Y}_{C,After}-\bar{Y}_{C,Before})$ The regression form in a two-group, two-period setting is: $Y_{it}=\alpha+\delta G_i+\lambda Post_t+\beta(G_i \times Post_t)+u_{it}$ With panel data, the common fixed effects version is: $Y_{it}=\alpha_i+\lambda_t+\beta D_{it}+u_{it}$ The central identifying assumption is parallel trends: $\mathbb{E}[Y_{After}(0)-Y_{Before}(0) \mid G=1] = \mathbb{E}[Y_{After}(0)-Y_{Before}(0) \mid G=0]$ In words, the treated and control groups would have followed the same trend in the absence of treatment. DiD can handle permanent differences between groups and common time shocks. But it can fail because of differential pre-trends, endogenous treatment timing, anticipation, spillovers, composition changes, simultaneous policies, staggered adoption problems, and incorrect inference. Event studies help examine pre-trends and treatment dynamics, but they do not prove the identifying assumption. Staggered adoption designs require special care because traditional two-way fixed effects estimates may be difficult to interpret under heterogeneous treatment effects. The central lesson is:

Difference-in-Differences is a counterfactual design. It is credible when the control group’s change over time is a plausible estimate of how the treated group would have changed without treatment.

12. Regression Discontinuity

12.1 What regression discontinuity is trying to do

Regression discontinuity, often abbreviated RD or RDD, is a quasi-experimental research design used when treatment assignment changes sharply at a known cutoff. The basic idea is simple:

If units just below and just above a cutoff are otherwise similar, then any discontinuous jump in the outcome at the cutoff can be interpreted as the causal effect of treatment for units near that cutoff.

RD is especially useful when institutions assign treatment using rules. Examples include:

students receive scholarships if test scores exceed a threshold,
households receive benefits if income falls below an eligibility cutoff,
firms face regulation if employment exceeds a size threshold,
students are placed in remedial education if exam scores fall below a cutoff,
politicians win elections if their vote share exceeds 50 percent,
patients receive treatment if a risk score crosses a clinical threshold,
schools receive funding if poverty rates exceed a program cutoff.

In all of these settings, treatment is not randomly assigned by the researcher. But the cutoff creates a discontinuous change in treatment probability. If units near the cutoff cannot precisely manipulate which side they fall on, then units just below and just above the threshold may be comparable. The key logic is local comparison. RD does not compare all treated units to all untreated units. It compares units near the cutoff. For example, suppose students with exam scores below 60 are assigned to tutoring, while students with scores above 60 are not. Students scoring 59 and 61 are likely more similar to each other than students scoring 30 and 95. If test scores are measured with some noise and students cannot precisely choose their score, then crossing the cutoff may approximate a local experiment. The causal question is:

What is the effect of treatment for units whose treatment status changes because they are just above or just below the cutoff?

The answer is local. It applies most directly to units near the threshold, not necessarily to all units in the population.

12.2 The running variable and the cutoff

RD requires a running variable, sometimes called an assignment variable, forcing variable, or score. Let: $X_i$ be the running variable for unit $i$. Let: $c$ be the cutoff. Treatment assignment depends on whether $X_i$ crosses $c$. For example:

$X_i$ could be a test score,
$c$ could be a scholarship eligibility threshold,
$D_i=1$ could mean the student receives a scholarship.

A simple treatment rule is: $D_i = 1 \quad \text{if} \quad X_i \ge c$ and $D_i = 0 \quad \text{if} \quad X_i < c$ In words: units at or above the cutoff are treated, and units below the cutoff are untreated. Sometimes treatment is assigned below the cutoff instead. For example, students below a test-score threshold may receive remedial tutoring: $D_i = 1 \quad \text{if} \quad X_i < c$ and $D_i = 0 \quad \text{if} \quad X_i \ge c$ The direction does not matter conceptually. What matters is that treatment changes discontinuously at a known threshold. It is often useful to center the running variable at the cutoff: $R_i = X_i - c$ Then the cutoff becomes zero. Units with $R_i < 0$ are on one side of the threshold, and units with $R_i \ge 0$ are on the other side. Centering makes interpretation easier because the treatment effect is estimated at $R_i=0$.

12.3 Sharp regression discontinuity

A sharp RD occurs when treatment status is completely determined by the cutoff. For example: $D_i = 1(X_i \ge c)$ where $1(\cdot)$ is an indicator function equal to one when the condition is true and zero otherwise. In a sharp RD, everyone on one side of the cutoff receives treatment, and everyone on the other side does not. The potential outcomes are: $Y_i(1)$ and $Y_i(0)$ where:

$Y_i(1)$ is the outcome if treated,
$Y_i(0)$ is the outcome if untreated.

The observed outcome is: $Y_i = D_iY_i(1) + (1-D_i)Y_i(0)$ The sharp RD estimand is the discontinuous jump in the conditional expectation of $Y_i$ at the cutoff: $\tau_{RD} = \lim_{x \downarrow c} \mathbb{E}[Y_i \mid X_i=x] - \lim_{x \uparrow c} \mathbb{E}[Y_i \mid X_i=x]$ if treatment begins above the cutoff. Here:

$\lim_{x \downarrow c}$ means the limit as $x$ approaches $c$ from above,
$\lim_{x \uparrow c}$ means the limit as $x$ approaches $c$ from below.

In plain English:

Compare the expected outcome just above the cutoff to the expected outcome just below the cutoff.

If potential outcomes would have changed smoothly through the cutoff in the absence of treatment, then any jump in the observed outcome at the cutoff can be attributed to treatment.

12.4 The continuity assumption

The core identifying assumption in RD is continuity of potential outcomes at the cutoff. For a sharp RD, the assumption is that: $\lim_{x \downarrow c} \mathbb{E}[Y_i(0) \mid X_i=x] = \lim_{x \uparrow c} \mathbb{E}[Y_i(0) \mid X_i=x]$ and: $\lim_{x \downarrow c} \mathbb{E}[Y_i(1) \mid X_i=x] = \lim_{x \uparrow c} \mathbb{E}[Y_i(1) \mid X_i=x]$ In words:

In the absence of the treatment discontinuity, expected potential outcomes would have evolved smoothly through the cutoff.

This does not mean that outcomes must be flat. Outcomes may vary with the running variable. Students with higher test scores may have higher later earnings. Firms with more employees may have higher revenue. Households with higher income may have different health outcomes. RD allows the outcome to depend on the running variable. The assumption is only that there would be no sudden jump at the exact cutoff except through treatment. For example, suppose students with scores below 60 receive tutoring. Students with scores of 59 and 61 may differ slightly in ability, but they are unlikely to differ dramatically. If later test scores jump sharply at 60, and nothing else changes at 60, that jump can be interpreted as the local effect of tutoring. The continuity assumption is not directly testable for potential outcomes because we do not observe both potential outcomes on both sides of the cutoff. But it has testable implications. Researchers often check whether pre-treatment covariates are continuous at the cutoff. If predetermined characteristics jump at the threshold, that suggests units just above and below the cutoff may not be comparable.

12.5 The local nature of RD

RD identifies a treatment effect at the cutoff. The estimand is local: $\tau_{RD} = \mathbb{E}[Y_i(1)-Y_i(0) \mid X_i=c]$ under the usual sharp RD assumptions. This means the estimated effect applies to units at or near the threshold. For example, a scholarship RD at a GPA cutoff of 3.5 estimates the effect of scholarship eligibility for students near GPA 3.5. It does not automatically estimate the effect for students with GPAs of 2.0 or 4.0. A tutoring RD at a test-score cutoff of 60 estimates the effect for students near 60. It does not necessarily apply to students who scored 20 or 95. This is both a strength and a limitation. It is a strength because local comparisons near the cutoff can be highly credible. It is a limitation because external validity may be narrow. The main question is:

Are units near the cutoff the policy-relevant population?

Sometimes they are. If a program is specifically designed for marginally eligible units, the RD estimand is highly relevant. But if policymakers want to know the effect for all eligible units, RD may provide only part of the answer.

12.6 Fuzzy regression discontinuity

A fuzzy RD occurs when the probability of treatment changes discontinuously at the cutoff, but treatment is not perfectly determined by the cutoff. For example, suppose students below a test-score threshold are eligible for tutoring, but not all eligible students attend. Some students above the cutoff may also receive tutoring through other channels. Then treatment probability jumps at the cutoff, but not from zero to one. Formally, the discontinuity in treatment probability is: $\lim_{x \downarrow c} \mathbb{E}[D_i \mid X_i=x] - \lim_{x \uparrow c} \mathbb{E}[D_i \mid X_i=x] \neq 0$ but the jump may be less than one. In fuzzy RD, crossing the cutoff can be interpreted as an instrument for treatment. Let: $Z_i = 1(X_i \ge c)$ where $Z_i$ indicates eligibility or assignment based on the cutoff. The fuzzy RD estimand is a ratio: $\tau_{FRD} = \frac{ \lim_{x \downarrow c} \mathbb{E}[Y_i \mid X_i=x] - \lim_{x \uparrow c} \mathbb{E}[Y_i \mid X_i=x] }{ \lim_{x \downarrow c} \mathbb{E}[D_i \mid X_i=x] - \lim_{x \uparrow c} \mathbb{E}[D_i \mid X_i=x] }$ The numerator is the jump in the outcome at the cutoff. The denominator is the jump in treatment probability at the cutoff. This is analogous to a local Wald estimator. In words:

Divide the discontinuity in the outcome by the discontinuity in treatment take-up.

Fuzzy RD estimates a local average treatment effect for compliers near the cutoff: units whose treatment status changes because they cross the threshold. This is similar to instrumental variables. The cutoff must affect the outcome through treatment, and not through other discontinuous channels.

12.7 RD as a local experiment

RD is often described as a local randomized experiment. The intuition is that, near the cutoff, small differences in the running variable may be partly arbitrary or noisy. Units just below and just above the threshold may be similar in observed and unobserved characteristics. For example, a student scoring 59.8 and a student scoring 60.2 may be very similar. If the cutoff is 60, one receives treatment and the other does not. The treatment difference is created by the threshold rule, not by a large underlying difference between the students. This local-experiment intuition is strongest when:

the running variable is continuous,
units cannot precisely manipulate the running variable,
the cutoff rule is strictly applied,
no other policy changes at the same cutoff,
units close to the cutoff are similar in predetermined characteristics,
outcomes are measured consistently on both sides.

However, RD is not literally randomized unless treatment near the cutoff is assigned by chance. The design is quasi-experimental. Its credibility depends on institutional knowledge and diagnostics. The phrase “local randomization” is sometimes used for RD designs that treat a narrow window around the cutoff as approximately randomized. This can be useful, but it requires justification. One must explain why units within that window are plausibly exchangeable.

12.8 Estimation with local regression

In practice, researchers estimate RD effects by fitting regression functions on each side of the cutoff. A simple local linear specification is: $Y_i = \alpha + \tau D_i + \beta_1(X_i-c) + \beta_2D_i(X_i-c) + u_i$ where:

$D_i=1(X_i \ge c)$,
$X_i-c$ is the centered running variable,
$D_i(X_i-c)$ allows the slope to differ on each side of the cutoff,
$\tau$ is the estimated jump at the cutoff.

This model fits one line below the cutoff and another line above the cutoff. The treatment effect is the vertical gap between the two lines at the cutoff. Researchers usually estimate the model using observations within a bandwidth around the cutoff: $|X_i-c| \le h$ where $h$ is the bandwidth. The bandwidth determines how close to the cutoff observations must be to enter the estimation sample. A small bandwidth uses observations very close to the cutoff. This improves comparability but may reduce precision. A large bandwidth uses more observations. This improves precision but may introduce bias if units far from the cutoff are less comparable or if the regression function is misspecified. This is the central bias-variance tradeoff in RD.

12.9 Bandwidth choice

Bandwidth choice is one of the most important practical decisions in RD. The bandwidth determines the estimation window around the cutoff. Let: $h>0$ be the bandwidth. The estimation sample is: $\{i: c-h \le X_i \le c+h\}$ A smaller $h$ gives a more local comparison. A larger $h$ gives more data and smaller standard errors. But a larger $h$ may require stronger assumptions about the shape of the relationship between $X_i$ and $Y_i$. Researchers often report results for multiple bandwidths to show robustness. For example, if the cutoff is 60, one might estimate effects using: $55 \le X_i \le 65$ then $50 \le X_i \le 70$ and $45 \le X_i \le 75$ A credible RD result should not depend entirely on one arbitrary bandwidth choice. Modern RD practice often uses data-driven bandwidth selectors, such as mean squared error optimal bandwidths or coverage-error-rate optimal bandwidths. These methods formalize the tradeoff between bias and variance. Even when data-driven bandwidths are used, researchers should still show sensitivity to reasonable alternatives.

12.10 Polynomial order and functional form

RD estimation requires approximating the conditional expectation function near the cutoff. A common recommendation is to use local linear or local quadratic regressions rather than high-order global polynomials. A local linear model is: $Y_i = \alpha + \tau D_i + \beta_1(X_i-c) + \beta_2D_i(X_i-c) + u_i$ A local quadratic model adds squared terms: $Y_i = \alpha + \tau D_i + \beta_1(X_i-c) + \beta_2(X_i-c)^2 + \beta_3D_i(X_i-c) + \beta_4D_i(X_i-c)^2 + u_i$ High-order global polynomials can behave poorly near boundaries and can create artificial jumps or sensitivity to distant observations. For this reason, modern RD practice usually favors local polynomial methods with appropriate bandwidths. The goal is not to fit the entire outcome-running-variable relationship perfectly. The goal is to estimate the jump at the cutoff. A simple rule of thumb:

RD estimation should be local, transparent, and robust to reasonable choices of bandwidth and functional form.

12.11 Kernels and weighting

Local RD estimators often give more weight to observations closer to the cutoff. A kernel function determines how observations are weighted within the bandwidth. One common choice is the triangular kernel: $K(u) = (1-|u|)1(|u|\le 1)$ where $u = \frac{X_i-c}{h}$ The triangular kernel gives the most weight to observations at the cutoff and zero weight to observations outside the bandwidth. The intuition is that observations closest to the cutoff are most comparable and most informative about the discontinuity. Other kernels exist, such as uniform or Epanechnikov kernels. In many applications, bandwidth choice matters more than kernel choice.

12.12 Graphical analysis

A good RD analysis usually includes a graph. The graph should show the relationship between the outcome and the running variable on both sides of the cutoff. A typical RD plot:

places the running variable on the horizontal axis,
places the outcome on the vertical axis,
marks the cutoff with a vertical line,
shows binned averages of the outcome,
overlays fitted regression lines on each side of the cutoff.

The purpose of the graph is not merely decorative. It helps readers see whether there is a visible discontinuity and whether the fitted model is plausible. A strong RD graph often shows:

a clear jump in the outcome at the cutoff,
smooth trends away from the cutoff,
no obvious outliers driving the result,
no strange functional-form artifacts,
reasonable density of observations near the cutoff.

A weak RD graph may show:

no visible jump,
a jump created only by an unusual polynomial fit,
sparse data near the cutoff,
large outliers near the threshold,
different trends that make extrapolation suspicious.

Graphical evidence is not sufficient by itself, but it is an important diagnostic.

12.13 No precise manipulation

A central threat to RD is manipulation of the running variable. The design is credible if units cannot precisely choose which side of the cutoff they fall on. If units can manipulate the running variable, then those just above and below the cutoff may not be comparable. For example:

students may retake or manipulate exams to cross a scholarship cutoff,
firms may keep employment just below a regulatory threshold,
households may underreport income to qualify for benefits,
politicians or officials may manipulate vote counts,
doctors may adjust risk scores to qualify patients for treatment.

When manipulation occurs, treatment status near the cutoff may reflect strategic behavior rather than quasi-random assignment. One common diagnostic is a density test. If many observations bunch on one side of the cutoff, that may indicate manipulation. The logic is:

If units cannot precisely control the running variable, the density of observations should be smooth through the cutoff.

A discontinuity in the density of $X_i$ at $c$ suggests that units may be sorting around the threshold. This does not automatically invalidate the design, but it is a serious warning sign.

12.14 Covariate balance at the cutoff

Another diagnostic is to test whether predetermined covariates are continuous at the cutoff. Let $W_i$ be a pre-treatment covariate, such as age, gender, prior income, baseline test score, parental education, firm age, or prior employment. Researchers can estimate RD-style discontinuities in $W_i$: $W_i = \alpha + \delta D_i + f(X_i-c) + u_i$ If $\delta$ is large or statistically significant for many predetermined variables, this suggests units on either side of the cutoff may differ for reasons other than treatment. In a credible RD, pre-treatment covariates should generally evolve smoothly through the cutoff. This is not because covariate balance proves the continuity assumption. The continuity assumption concerns unobserved potential outcomes. But covariate balance supports the idea that the cutoff is not associated with abrupt changes in unit characteristics. For example, in a scholarship RD, students just above and below the score cutoff should have similar prior grades, demographics, family background, and school characteristics. If parental income jumps sharply at the cutoff, that would raise concerns.

12.15 Placebo cutoffs and placebo outcomes

Researchers often conduct placebo tests. A placebo cutoff test estimates whether there are discontinuities at values of the running variable where no treatment rule changes. For example, if the actual cutoff is 60, one might test for jumps at 50 or 70. If there are large jumps at many fake cutoffs, then the observed jump at the true cutoff may not be caused by treatment. It may reflect irregularities in the outcome function or model misspecification. A placebo outcome test uses an outcome that should not be affected by treatment. For example, if a scholarship is awarded after the exam, it should not affect a student’s pre-treatment grades. If pre-treatment grades jump at the scholarship cutoff, that suggests selection or manipulation. Placebo tests do not prove validity, but they can reveal problems. A good RD design should show a discontinuity in variables that treatment should affect and no discontinuity in variables that treatment could not plausibly affect.

12.16 Donut regression discontinuity

A donut RD excludes observations very close to the cutoff. This may seem counterintuitive because observations close to the cutoff are usually most valuable. But donut RD can be useful if observations extremely close to the cutoff are suspicious. For example, manipulation may occur only near the threshold. Firms may carefully adjust employment to stay just below a regulation cutoff. Students just below a cutoff may appeal grades. Households near an income threshold may report income strategically. In a donut RD, researchers remove observations within a small window around the cutoff: $|X_i-c| < \epsilon$ and estimate the RD using observations slightly farther away. If results are robust to excluding the immediate neighborhood of the cutoff, that may reduce concern about manipulation or measurement error at the threshold. However, donut RD also makes the comparison less local. It should be used as a sensitivity analysis, not as a default substitute for understanding the assignment process.

12.17 Discrete running variables

Many RD designs assume the running variable is continuous or has many possible values. But sometimes the running variable is discrete. Examples:

age in years,
test scores rounded to integers,
income reported in bins,
firm size measured in whole employees,
vote margin measured in percentage points but rounded,
school poverty rates rounded to whole numbers.

When the running variable is discrete, there may be few distinct values near the cutoff. This makes it harder to estimate limits from above and below. A discrete running variable can create problems:

fewer comparison points near the cutoff,
sensitivity to functional form,
difficulty using conventional bandwidth methods,
clustering of observations at score values,
imperfect local comparability.

RD can still be useful with discrete running variables, but inference and interpretation require caution. Researchers should report how many support points exist near the cutoff and avoid pretending that a coarse running variable provides the same information as a continuous one.

12.18 Multiple cutoffs

Some RD designs involve multiple cutoffs. Examples:

different states use different income thresholds for benefits,
schools use grade-specific test-score cutoffs,
programs use different eligibility rules by region,
tax schedules use multiple brackets,
regulations apply at several firm-size thresholds.

With multiple cutoffs, researchers must be clear about what effect is being estimated. One possibility is to estimate separate RD effects at each cutoff. Another is to pool across cutoffs after centering each running variable around its cutoff. If effects differ across cutoffs, a pooled estimate may average different local treatment effects. This can be useful, but it requires interpretation:

The estimate is an average of local effects at the included thresholds, not necessarily a universal treatment effect.

Multiple cutoffs can also improve external validity if they identify effects for different types of units. But they can complicate estimation if assignment rules differ across settings.

12.19 Geographic regression discontinuity

A geographic RD uses a border or boundary as the cutoff. Examples:

one side of a state border has a different minimum wage,
one school district has a different funding rule,
one municipality has a different tax policy,
one side of a pollution regulation boundary faces stricter rules,
one administrative region receives a program while a neighboring region does not.

The running variable may be distance to the border. The logic is that units close to each other geographically may be similar except for the policy environment. For example, workers living just across a state border may face similar labor markets but different minimum wage laws. The key assumption is spatial continuity:

In the absence of the policy discontinuity, potential outcomes would vary smoothly across the boundary.

Threats include:

sorting across the border,
other policies changing at the same border,
geographic discontinuities in demographics,
differences in local institutions,
commuting and spillovers,
housing market sorting,
historical reasons the border matters.

Geographic RD can be powerful, but borders often bundle many differences. Researchers must show that the treatment of interest, not some other border discontinuity, explains the outcome jump.

12.20 RD with time and policy thresholds

Some RD designs use time as the running variable. For example, a policy begins on a specific date. One might compare outcomes just before and just after the policy implementation date. This is sometimes called regression discontinuity in time. Let $T_i$ be time, and let $c$ be the policy date. Treatment turns on after the cutoff: $D_i = 1(T_i \ge c)$ The problem is that time itself often causes outcomes to change. Trends, seasonality, anticipation, and other events can create discontinuities or changes near the policy date. RD in time is usually less credible than RD based on a cross-sectional assignment rule unless the timing is plausibly unrelated to other shocks and outcomes would have evolved smoothly at that date. For example, a tax change implemented on January 1 may coincide with seasonal changes, fiscal-year changes, or other policies. To strengthen RD in time, researchers often use:

narrow time windows,
seasonality controls,
placebo dates,
unaffected comparison groups,
event-study plots,
institutional details about policy timing.

The key question remains:

Would the outcome have been continuous through the policy date absent the treatment?

12.21 RD versus difference-in-differences

RD and DiD are both quasi-experimental designs, but they rely on different comparisons. RD compares units near a cutoff. DiD compares changes over time between treated and control groups. RD relies on continuity at the threshold. DiD relies on parallel trends. RD is often strongest when treatment changes abruptly according to a rule. DiD is often strongest when treatment changes for some units but not others over time. The two designs can sometimes be combined. For example, suppose a policy cutoff exists before and after a reform. A researcher may compare discontinuities before and after the policy change. This is sometimes called difference-in-discontinuities. The goal is to remove pre-existing jumps at the cutoff and isolate the change in the discontinuity caused by the reform. The choice between RD and DiD depends on the source of identifying variation. Ask:

Is treatment assigned by a threshold rule?
Are units near the threshold comparable?
Is there a control group over time?
Are pre-treatment trends parallel?
Does the cutoff itself create the best counterfactual?
Does time variation create the best counterfactual?

12.22 RD and instrumental variables

Fuzzy RD is closely connected to instrumental variables. In fuzzy RD, crossing the cutoff affects treatment probability but does not perfectly determine treatment. The cutoff indicator: $Z_i = 1(X_i \ge c)$ serves as an instrument for treatment $D_i$. The first stage is the jump in treatment probability: $\lim_{x \downarrow c} \mathbb{E}[D_i \mid X_i=x] - \lim_{x \uparrow c} \mathbb{E}[D_i \mid X_i=x]$ The reduced form is the jump in outcomes: $\lim_{x \downarrow c} \mathbb{E}[Y_i \mid X_i=x] - \lim_{x \uparrow c} \mathbb{E}[Y_i \mid X_i=x]$ The fuzzy RD estimate is: $\frac{Reduced\ Form}{First\ Stage}$ This estimates the treatment effect for compliers near the cutoff. The IV assumptions become local RD assumptions:

the cutoff must affect treatment probability,
potential outcomes must be continuous at the cutoff,
the cutoff must affect the outcome only through treatment,
treatment must move monotonically for units near the cutoff.

The exclusion restriction in fuzzy RD means there should be no other discontinuous change at the cutoff that affects the outcome.

12.23 Common RD examples

Example	RD logic	Possible outcomes	Main threats
Scholarships and test-score cutoffs	Students just above a score threshold receive aid; students just below do not.	College enrollment, graduation, earnings, debt, field of study.	Score manipulation, other awards at the same cutoff, inconsistent grading, unobserved differences near the threshold, and effects applying only near the cutoff.
Remedial education	Students below a placement-test cutoff are assigned to remediation; students just above are not.	Later math achievement, college credits, graduation, dropout.	Retesting, fuzzy compliance, stigma or tracking effects, and other academic services changing at the same cutoff.
Close elections	The running variable is vote margin; barely winning is compared with barely losing, for example $Vote\ Share_i \ge 0.5$.	Future policy, public spending, economic growth, reelection, legislative behavior.	Manipulation or fraud, recount rules, different campaign strategies, and limited external validity beyond close elections.
Program eligibility by income	Households just below an income threshold qualify for a subsidy; households just above do not.	Consumption, health care use, labor supply, savings, educational investment.	Income manipulation, measurement error, multiple programs at the same cutoff, bunching below the threshold, and incomplete take-up.
Firm-size regulations	Firms just above an employment threshold face a rule; firms just below do not.	Employment growth, wages, productivity, compliance costs, firm exit.	Firms manipulating headcount, discrete running variables, multiple laws at the same threshold, unusual firms near the cutoff, and anticipatory behavior.

2.24 Common mistakes in RD.

Mistake	Why it matters
Comparing all treated and untreated units	RD is a local comparison near the cutoff, not a comparison of all treated and untreated units.
Ignoring manipulation	If units precisely sort around the threshold, the cutoff comparison may not be credible.
Treating the estimate as global	RD estimates a local effect at the cutoff; external validity beyond that margin must be argued.
Using high-order global polynomials uncritically	High-order polynomials can create misleading fits and sensitivity to distant observations; local methods are usually safer.
Ignoring other discontinuities	If another policy or incentive changes at the same threshold, the RD estimate may combine multiple effects.
Failing to show the graph	RD credibility depends heavily on whether the discontinuity is visible and the fitted functions are plausible.
Ignoring treatment take-up	If treatment does not perfectly change at the cutoff, the design is fuzzy rather than sharp.
Using covariates mechanically	Covariates can improve precision, but RD identification should not depend on adding many controls.

12.25–12.26 RD evaluation and application checklist.

Dimension	Questions to answer
Causal question	What treatment effect is being estimated, and what policy, program, or exposure changes at the cutoff?
Unit of analysis	Are the units individuals, firms, schools, hospitals, neighborhoods, elections, or jurisdictions?
Running variable	What variable determines treatment assignment? Is it measured before treatment? Is it continuous, discrete, or coarse?
Cutoff	What is the threshold? Why does it exist? Was it chosen by policy, institution, or the researcher?
Treatment jump	Is the design sharp or fuzzy? Does treatment probability jump clearly at the cutoff?
Identifying assumption	For sharp RD, are potential outcomes continuous at the cutoff? For fuzzy RD, is the cutoff a valid local instrument?
Manipulation	Can units precisely control the running variable? Is there bunching near the threshold?
Covariate balance	Do predetermined covariates evolve smoothly through the cutoff, or are there suspicious jumps?
Other discontinuities	Do other policies, incentives, or institutional rules change at the same threshold?
Graphical evidence	Do binned means and fitted functions make the discontinuity and identifying variation transparent?
Bandwidth and functional form	Are results robust to narrower/wider bandwidths and reasonable local specifications? Are high-order global polynomials avoided?
Placebo checks	Are fake cutoffs and placebo outcomes examined?
Estimand and interpretation	Is the estimate a sharp RD effect, fuzzy RD/LATE, eligibility effect, or treatment take-up effect? Does it apply only near the cutoff?
External validity	Are units near the cutoff policy-relevant? Would effects differ far from the threshold or under scaled eligibility rules?

12.27 Summary

Regression discontinuity is a quasi-experimental design based on threshold rules. A running variable $X_i$ determines whether a unit is above or below a cutoff $c$. In a sharp RD, treatment is fully determined by the cutoff: $D_i = 1(X_i \ge c)$. The sharp RD estimand is the jump in the conditional expectation of the outcome at the cutoff: $\tau_{RD} = \lim_{x \downarrow c} \mathbb{E}[Y_i \mid X_i=x] - \lim_{x \uparrow c} \mathbb{E}[Y_i \mid X_i=x]$ In a fuzzy RD, treatment probability jumps at the cutoff but does not change perfectly from zero to one. The fuzzy RD estimand is the ratio of the outcome discontinuity to the treatment discontinuity: $\tau_{FRD} = \frac{ \text{Jump in outcome at cutoff} }{ \text{Jump in treatment probability at cutoff} }$ The core RD assumption is continuity of potential outcomes at the cutoff. In the absence of treatment, outcomes would have changed smoothly through the threshold. RD is powerful because it can create credible local comparisons using institutional rules. But it is credible only when units cannot precisely manipulate the running variable, no other important factor changes discontinuously at the cutoff, and the comparison remains local. The main strengths of RD are:

transparent assignment rules,
strong local internal validity,
intuitive graphical presentation,
clear diagnostics,
close connection to institutional design.

The main limitations are:

local external validity,
sensitivity to bandwidth and functional form,
vulnerability to manipulation,
problems with discrete running variables,
possible confounding from other discontinuities at the same cutoff.

The central lesson is:

Regression discontinuity estimates causal effects by comparing units just on either side of a treatment threshold. Its credibility comes from the idea that potential outcomes would have been smooth through the cutoff if treatment had not changed discontinuously there.

Group	Before	After	Change
Treated	\(\bar{Y}_{T,Before}\)	\(\bar{Y}_{T,After}\)	\(\bar{Y}_{T,After}-\bar{Y}_{T,Before}\)
Control	\(\bar{Y}_{C,Before}\)	\(\bar{Y}_{C,After}\)	\(\bar{Y}_{C,After}-\bar{Y}_{C,Before}\)

Group	Before	After	Change
Treated state	100	95	\(95-100=-5\)
Control state	100	98	\(98-100=-2\)

Core Concepts in Economics: Empirics Part 2

Core Concepts in Empirical Economics and Causal Inference, Part 2

Table of Contents

Introduction to Part 2

7. Randomized Controlled Trials, or RCTs

7.1 Why randomized controlled trials matter

7.2 Treatment, control, and random assignment

7.3 The basic RCT estimand

7.4 Why randomization solves confounding

7.5 Randomization in expectation versus balance in a sample

7.6 Estimation using regression

7.7 Intent-to-treat effects

7.8 Treatment-on-the-treated and noncompliance

7.9 Types of noncompliance

7.10 Attrition

7.11 Spillovers and interference

7.12 Hawthorne effects, John Henry effects, and placebo effects

7.13 Internal validity in RCTs

7.14 External validity in RCTs

7.15 Individual-level randomization

7.16 Cluster randomization

7.17 Encouragement designs

7.18 Phase-in designs

7.19 Factorial designs

7.20 Power and minimum detectable effects

7.21 Multiple outcomes and multiple testing

7.22 Heterogeneous treatment effects in RCTs

7.23 Ethical issues in RCTs

7.24 Practical example: job training RCT

7.25 Practical example: education RCT

7.26 Practical example: information experiment

7.30 Summary

8. Natural Experiments and Quasi-Experimental Design

8.1 Why natural experiments matter

8.2 Natural experiments versus ordinary observational studies

8.3 The experimental ideal and the quasi-experimental substitute

8.4 What makes a natural experiment credible?

8.5 As-good-as-random variation

8.6 Common sources of natural-experimental variation

8.7 Quasi-experimental designs

8.8 Difference-in-differences as a quasi-experiment

8.9 Regression discontinuity as a quasi-experiment

8.10 Instrumental variables as a quasi-experiment

8.11 Synthetic control

8.12 Event studies

8.13 Interrupted time series

8.14 Matching and weighting as quasi-experimental tools

8.15 Panel fixed effects as a quasi-experimental tool

8.16 The role of institutional knowledge

8.17 Diagnostics and falsification tests

8.18 Common threats to quasi-experimental validity

8.19 Identification, estimation, and inference in quasi-experiments

8.20 Practical example: compulsory schooling laws

8.21 Practical example: minimum wage changes

8.22 Practical example: judge assignment and incarceration

8.23 Practical example: weather shocks and agricultural income

8.27 Summary

9. Identification Strategy

9.1 Why identification is central to empirical economics

9.2 Identification versus estimation

9.3 Three meanings of identification

9.4 The anatomy of an identification strategy

9.5 A simple example: job training and earnings

9.6 Identification as a mapping from causal quantities to observed data

9.7 The role of assumptions

9.8 Exogenous variation

9.9 Identifying variation versus control variation

9.10 Identification in randomized experiments

9.11 Identification in difference-in-differences

9.12 Identification in regression discontinuity

9.13 Identification in instrumental variables

9.14 Identification in fixed effects designs

9.15 Identification in matching and weighting

9.16 Identification in synthetic control designs

9.17 Identification and DAGs

9.18 Identification and bad controls

9.19 Local versus global identification

9.20 Identification and heterogeneity

9.21 Identification and external validity

9.22 How researchers defend identification strategies