Journal Article

STRENGTHENING THE REGRESSION DISCONTINUITY DESIGN USING ADDITIONAL DESIGN ELEMENTS: A WITHIN-STUDY COMPARISON

Cook, Thomas D., Wing, Coady

2013

Overview

The sharp regression discontinuity design (RDD) has three key weaknesses compared to the randomized clinical trial (RCT). It has lower statistical power, it is more dependent on statistical modeling assumptions, and its treatment effect estimates are limited to the narrow subpopulation of cases immediately around the cutoff, which is rarely of direct scientific or policy interest. This paper examines how adding an untreated comparison to the basic RDD structure can mitigate these three problems. In the example we present, pretest observations on the posttest outcome measure are used to form a comparison RDD function. To assess its performance as a supplement to the basic RDD, we designed a within-study comparison that compares causal estimates and their standard errors for (1) the basic posttest-only RDD, (2) a pretest-supplemented RDD, and (3) an RCT chosen to serve as the causal benchmark. The two RDD designs are constructed from the RCT, and all analyses are replicated with three different assignment cutoffs in three American states. The results show that adding the pretest makes functional form assumptions more transparent. It also produces causal estimates that are more precise than in the posttest-only RDD, but that are nonetheless larger than in the RCT. Neither RDD version shows much bias at the cutoff, and the pretest-supplemented RDD produces causal effects in the region beyond the cutoff that are very similar to the RCT estimates for that same region. Thus, the pretest-supplemented RDD improves on the standard RDD in multiple ways that bring causal estimates and their standard errors closer to those of an RCT, not just at the cutoff, but also away from it.

• Publisher: Blackwell Publishing Ltd,Wiley Subscription Services, Inc,Wiley-Blackwell,Wiley Periodicals Inc
• Subject: Age / Benchmarking / Causal Models / Causality / Clinical research / Clinical trials / Comparative Analysis / Comparative Testing / Cutting Scores / Design / Discontinuity / Error / Error of Measurement / Errors / Estimating techniques / Estimation bias / Estimation methods / Linear regression / Mathematical extrapolation / Mathematical models / Medicaid / Medical Research / Methods for Policy Analysis / Outcome Measures / Policy analysis / Policy studies / Political science / Polynomials / Pretests Posttests / Public policy / Randomized Controlled Trials / Regions / Regression (Statistics) / Regression analysis / Research Design / Research Problems / Statistical Analysis / Statistical estimation / Statistical power / Studies / U.S.A