Propensity Score Matching

Propensity Score Matching is a statistical method for estimating the causal effect of a treatment, exposure, or program from observational data by comparing treated and untreated units that had the same estimated probability of receiving treatment. Introduced by Paul Rosenbaum and Donald Rubin in a 1983 paper in Biometrika, it reduces a high-dimensional set of measured covariates to a single scalar, the propensity score e(X) = Pr(T = 1 | X), and uses it to construct comparison groups that resemble those of a randomized controlled trial with respect to the measured covariates. In public health, the method is used to evaluate vaccination programs, drugs in routine clinical care, screening, and policy interventions where randomization is impractical or unethical. Its popularity rests on intuitive appeal and on a clear theoretical result, while its limitations have prompted sustained debate.

The framework is the Rubin causal model. Each unit i has two potential outcomes, Y_i(1) under treatment and Y_i(0) under control, and only one is observed. The average treatment effect is ATE = E[Y(1) − Y(0)], and the average effect on the treated is ATT = E[Y(1) − Y(0) | T = 1]. Naive comparison of observed group means is biased when covariates that affect the outcome also affect who is treated, a problem known as confounding in its clinical form.

Two assumptions justify propensity-based estimation. Strong ignorability (unconfoundedness) requires that treatment assignment is independent of the potential outcomes given the measured covariates: (Y(1), Y(0)) ⫫ T | X. Positivity (overlap) requires 0 < e(X) < 1 for all covariate values, so that every kind of unit has some chance of both treatment and control. Together they mean that within strata of X the data behave like a randomized experiment.

Rosenbaum and Rubin proved that the propensity score is the coarsest balancing score: conditional on e(X), the distribution of X is the same in treated and control groups, so X ⫫ T | e(X). They also showed that if treatment assignment is ignorable given X, it is ignorable given e(X) alone. The consequence is dimension reduction. Instead of matching on dozens of covariates, an analyst can match on one number and still balance all covariates, in expectation, that enter the score.

The true score is unknown and is estimated, usually by logistic regression of treatment on covariates, though classification trees, boosting, and neural networks are also used. Balance, not predictive accuracy, is the target: a score model that predicts treatment well may still fail to balance covariates, and a poorly predictive model may balance well. Variables associated with the outcome belong in the score model even if they only weakly predict treatment, since they reduce variance; variables that only predict treatment (instruments) should be left out, because they inflate variance and can amplify bias from unmeasured confounding, a concern related to the structure discussed in collider bias.

Nearest-neighbor matching pairs each treated unit with the control whose score is closest, with or without replacement. Matching with replacement lowers bias but reuses controls, which must be accounted for in the variance. Caliper matching adds a maximum allowed distance, commonly 0.2 standard deviations of the logit of the score as recommended by Austin, and discards treated units without a close control, trading generalizability for balance. Optimal and full matching minimize total distance across all pairs or form variable-size matched sets. Kernel and radius matching use weighted averages of many controls. Stratification divides the score distribution into quintiles or finer strata and averages within-stratum differences; Rosenbaum and Rubin noted that five strata remove most of the bias attributable to a covariate. Inverse probability weighting is a related alternative, assigning weights 1/e(X) to treated and 1/(1 − e(X)) to controls.

The estimate is the average outcome difference over matched sets. Standard errors are not straightforward because the score is estimated and matching creates dependence. Bootstrapping is invalid for some matching estimators, as shown by Abadie and Imbens, so analytic variance estimators or matched-set methods are preferred.

The check of a propensity analysis is balance on covariates after matching, not the significance of the score model. The standard measure is the standardized mean difference, SMD = (mean_treated − mean_control) divided by the pooled standard deviation, with values below 0.1 conventionally regarded as acceptable. Significance tests of balance are discouraged because they depend on sample size. Variance ratios, comparisons of full distributions, and examination of interactions complement mean differences.

Overlap is assessed by plotting score distributions for the two groups. Regions of the score with only treated or only control units lack comparable counterparts, and estimation there relies on extrapolation. Trimming such units changes the estimand to the effect in the region of common support. Poor overlap is common in pharmacoepidemiology when a drug is prescribed only to patients with specific indications, and it is a signal that the question may not be answerable with the available data.

In 2019, Gary King and Richard Nielsen argued in Political Analysis that propensity score matching can increase imbalance, inefficiency, model dependence, and bias relative to other matching approaches. Their reasoning is that pruning observations to match on the score approximates a completely randomized experiment, in which units are balanced only on average, rather than a fully blocked experiment, in which units are exactly paired on covariates. As matching proceeds and the sample is pruned, random components of the score cause the method to drop units arbitrarily, so imbalance can rise rather than fall. They recommended matching directly on covariates, using Mahalanobis distance or coarsened exact matching, which approximate blocked designs.

Defenders respond that the critique mainly applies to aggressive pruning without balance checks, that propensity scores remain valuable for weighting and stratification, and that the method is sound when iteratively refined until balance is attained. The disagreement has changed practice: analysts report balance diagnostics prominently, avoid treating the score as a final answer, and often combine approaches.

Propensity methods and outcome regression each have a failure mode: the score model can be wrong, or the outcome model can be wrong. Doubly robust estimators such as augmented inverse probability weighting and targeted maximum likelihood combine the two and remain consistent if either is correctly specified. They have become a default in modern epidemiology, with propensity matching often used as a design stage and a regression adjustment performed on the matched sample.

No propensity analysis protects against unmeasured confounding. The E-value of VanderWeele and Ding quantifies how strong an unmeasured confounder, in terms of risk-ratio association with both treatment and outcome, would need to be to explain away an observed effect. Related approaches are catalogued under quantitative bias analysis, and negative-control designs, including proximal causal inference, offer ways to detect residual confounding. Genetic instruments, as in Mendelian randomization, supply a complementary route when ignorability is doubtful.

Good practice follows a sequence: define the target estimand, choose covariates measured before treatment, estimate the score, match, evaluate balance and overlap, estimate the effect, and perform sensitivity analysis. Propensity score matching clarifies the design of an epidemiology study by separating treatment modeling from outcome analysis, allowing balance to be judged without looking at outcomes. Its ceiling is set by the measured covariates and the plausibility of ignorability, not by the sophistication of the matching algorithm.

1. Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41–55. https://doi.org/10.1093/biomet/70.1.41
2. King, G., & Nielsen, R. (2019). Why propensity scores should not be used for matching. Political Analysis, 27(4), 435–454. https://doi.org/10.1017/pan.2019.11
3. Austin, P. C. (2011). An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behavioral Research, 46(3), 399–424. https://doi.org/10.1080/00273171.2011.568786
4. VanderWeele, T. J., & Ding, P. (2017). Sensitivity analysis in observational research: Introducing the E-value. Annals of Internal Medicine, 167(4), 268–274. https://doi.org/10.7326/M16-2607
5. Bang, H., & Robins, J. M. (2005). Doubly robust estimation in missing data and causal inference models. Biometrics, 61(4), 962–973. https://doi.org/10.1111/j.1541-0420.2005.00377.x