Difference-in-Differences

Difference-in-differences (abbreviated DiD or DD) is a quasi-experimental longitudinal research design in epidemiology, biostatistics, health economics, and public health policy analysis that estimates the Average Treatment Effect on the Treated (ATT) by contrasting the longitudinal change in outcomes from before to after an intervention in an exposed group (Treated) against the concurrent change over the exact same calendar periods in an unexposed comparison group (Control). Originating in John Snow's 1855 natural experiment comparing cholera mortality in London neighborhoods served by the Lambeth Waterworks (which relocated its Thames intake upstream of sewage outfalls between 1849 and 1854) against neighborhoods served by the Southwark and Vauxhall Waterworks (which retained its polluted downstream intake), and formalized in econometrics by Orley Ashenfelter (1978) and David Card and Alan B. Krueger (1994), the difference-in-differences estimator subtracts twice: the first difference (post minus pre within the treated group) removes all time-invariant between-group confounding, while the second difference (subtracting the post-minus-pre change in the control group) removes common calendar-time shocks and secular trends shared by both populations.

Double Subtraction of Time-Invariant and Period Effects

Consider two population groups indexed by g ∈ {T, C} (T = treated jurisdiction; C = control jurisdiction) observed across two calendar periods t ∈ {0, 1} (t = 0 pre-intervention baseline; t = 1 post-intervention follow-up), where treatment is administered only to group T in period t = 1 (D_gt = 1(g = T) × 1(t = 1)). Under Donald Rubin's potential-outcomes framework, let Y_igt(1) denote the potential outcome for individual i in group g at time t if exposed to the policy, and let Y_igt(0) denote the counterfactual outcome in the absence of the policy.

The target causal estimand is the Average Treatment Effect on the Treated in the post-policy period (ATT):

τ_ATT = E[Y_*iT*1(1) − Y_*iT*1(0) | G = T]

Because the counterfactual untreated outcome E[Y_*iT*1(0) | G = T] is unobserved in the post-intervention period, the 2 × 2 difference-in-differences estimator imputes the unobserved post-pre change of the treated group using the observed post-pre change of the control group:

τ̂_DiD = ( Ȳ_T,1 − Ȳ_T,0 ) − ( Ȳ_C,1 − Ȳ_C,0 )

In regression notation, τ̂_DiD is estimated by the interaction coefficient δ in the ordinary least squares model:

Y_igt = β₀ + β₁ Treated_g + β₂ Post_t + δ (Treated_g × Post_t) + γ^TX_igt + ε_igt

1. The Parallel Trends (Common Trends) Assumption

Unlike randomized controlled trials or propensity-score matching, difference-in-differences does not require that the treated and control groups have identical baseline outcome levels (Ȳ_T,0 ≠ Ȳ_C,0 is explicitly permitted and absorbed by β₁). Instead, unbiased identification of τ_ATT rests on the parallel trends assumption: in the counterfactual absence of the intervention, the average outcome in the treated group would have followed the exact same temporal trajectory from t = 0 to t = 1 as the average outcome in the control group:

E[Y_*iT*1(0) − Y_*iT*0(0) | G = T] = E[Y_*iC*1(0) − Y_*iC*0(0) | G = C]

Importantly, the parallel trends assumption is scale-dependent: if baseline outcome rates differ substantially between groups (for example, baseline infant mortality is 20 per 1,000 in group T versus 5 per 1,000 in group C), an equal multiplicative percentage decline (parallel trends on the log-rate Poisson/logit scale) implies unequal additive declines on the linear risk-difference scale. Epidemiologists therefore evaluate whether additive linear models or multiplicative log-linear Poisson regressions match the underlying biological kinetics.

2. No Anticipation and Stable Unit Treatment Value Assumption (SUTVA)

Two additional structural conditions are required:

• No Anticipatory Behavior: Individuals or clinicians in the treated jurisdiction must not alter their behavior prior to t = 1 in anticipation of the policy announcement (which would contaminate the baseline Ȳ_T,0).
• No Geographic Spillover (Interference): The policy enacted in the treated jurisdiction must not spill over to alter outcomes in the comparison jurisdiction (for example, patients crossing state borders to obtain subsidized care or vaccines).

Event-Study Lead/Lag Plots and Pre-Trend Falsification

When multi-period longitudinal data (t = −K, ..., −2, −1, 0, +1, ..., +L) are available, investigators fit a dynamic event-study specification replacing the single Treated × Post term with relative-time lead and lag interaction indicators (using k = −1, the period immediately preceding policy enactment, as the omitted reference period):

Y_igt = α_g + λ_t + ∑_k≠−1 δ_k · 1(t − E_g = k) + ε_igt

Plotting the estimated pre-treatment lead coefficients (δ_−K, ..., δ₋₂) provides an empirical falsification check of parallel pre-trends (all pre-policy leads should fluctuate near 0), while the post-treatment lag coefficients (δ₀, δ₁, ..., δ_L) reveal whether policy effects emerge immediately, accumulate gradually, or wane over time.

Negative-Weight Contamination in Differential Timing Rollouts

In many global health and state-level policy studies—such as staggered state Medicaid expansions under the US Affordable Care Act, sequential provincial rollout of rotavirus vaccination, or municipal sugar-sweetened beverage taxes—different jurisdictions adopt the intervention at different calendar times (E_g). Historically, researchers analyzed staggered rollouts using a static Two-Way Fixed Effects (TWFE) regression with unit fixed effects (α_g) and calendar-time fixed effects (λ_t):

Y_gt = α_g + λ_t + β_TWFE D_gt + ε_gt

Breakthrough econometric theorems by Andrew Goodman-Bacon (2021), Clément de Chaisemartin and Xavier D'Haultfœuille (2020), Brantly Callaway and Pedro H. C. Sant'Anna (2021), and Liyang Sun and Sarah Abraham (2021) proved that β_TWFE is a weighted average of all possible 2 × 2 DiD comparisons in the dataset, including forbidden comparisons that use already-treated early adopters as the "control" group for late adopters. Whenever treatment effects evolve dynamically over time (for example, a health policy whose mortality benefit grows over 3 to 5 years), the growing treatment effect inside the early-adopter "control" group is subtracted from the late-adopter group, assigning negative weights to valid cohort-time treatment effects and potentially flipping the sign of β_TWFE!

Modern Heterogeneity-Robust Staggered DiD Estimators

To eliminate forbidden comparisons, modern epidemiological studies replace static TWFE regressions with heterogeneity-robust group-time estimators:

• Callaway and Sant'Anna (2021) Group-Time ATT(g, t): Estimates separate cohort-specific treatment effects ATT(g, t) for units first treated at time g evaluated at calendar time t, strictly restricting the comparison group to either never-treated units or not-yet-treated units (E_c > t), and aggregates them by event time e = t − g with doubly robust inverse-probability weighting.
• Sun and Abraham (2021) Interaction-Weighted Estimator: Interacts relative-time indicators with treatment-cohort indicators so different adoption cohorts are never pooled with improper baseline weights.

Because policy interventions are assigned at the jurisdiction or hospital cluster level (G units) while outcomes are measured over multiple serially correlated time periods (T), standard OLS variance formulas severely overstate precision. As demonstrated by Marianne Bertrand, Esther Duflo, and Sendhil Mullainathan (2004), standard errors must be clustered at the level of policy assignment (cluster-robust sandwich variance) or computed via the wild cluster bootstrap when the number of treated and control jurisdictions is small (G < 30).

When a jurisdiction-specific shock coincides with the policy at t = 1, investigators can add an internal within-jurisdiction negative control subgroup (such as an age group ineligible for the policy) to form a Difference-in-Difference-in-Differences (Triple Difference, DDD) estimator, or construct a convex weighted comparison group via the synthetic control method.

What should an investigator do if pre-intervention trends are not parallel?

If pre-treatment event-study leads (δ_−k) show diverging trajectories between treated and control groups, a standard unconditional DiD estimator is biased. Investigators have three rigorous alternatives: (1) use doubly robust conditional DiD (Sant'Anna & Zhao, 2020) if trends are parallel after conditioning on baseline demographic covariates; (2) switch to a Controlled Interrupted Time Series model that explicitly accounts for differential linear pre-slopes; or (3) apply the synthetic control method or Synthetic Difference-in-Differences (Arkhangelsky et al., 2021) to reweight donor units so their pre-intervention trajectory matches the treated unit.

Why is adjusting for post-treatment covariates in a DiD regression dangerous?

Controlling for covariates measured after the policy takes effect (t ≥ 1) that are themselves affected by the intervention introduces post-treatment collider bias and overadjustment bias (blocking part of the causal pathway). Covariates in DiD models should be restricted to baseline pre-intervention characteristics (X_g,0) or exogenous environmental variables.

1. Wing, C., Simon, K., & Bello-Gomez, R. A. (2018). Designing difference in difference studies: best practices for public health policy research. Annual Review of Public Health, 39, 453–469. https://doi.org/10.1146/annurev-publhealth-040617-013507
2. Bertrand, M., Duflo, E., & Mullainathan, S. (2004). How much should we trust differences-in-differences estimates? The Quarterly Journal of Economics, 119(1), 249–275. https://doi.org/10.1162/003355304772839588
3. Goodman-Bacon, A. (2021). Difference-in-differences with variation in treatment timing. Journal of Econometrics, 225(2), 254–277. https://doi.org/10.1016/j.jeconom.2021.03.014
4. Callaway, B., & Sant'Anna, P. H. C. (2021). Difference-in-differences with multiple time periods. Journal of Econometrics, 225(2), 200–230. https://doi.org/10.1016/j.jeconom.2020.12.001