  To make informed policy recommendations from observational data, we must be
able to discern true treatment effects from random noise and effects due to
confounding. Difference-in-Difference techniques which match treated units to
control units based on pre-treatment outcomes, such as the synthetic control
approach have been presented as principled methods to account for confounding.
However, we show that use of synthetic controls or other matching procedures
can introduce regression to the mean (RTM) bias into estimates of the average
treatment effect on the treated. Through simulations, we show RTM bias can lead
to inflated type I error rates as well as decreased power in typical policy
evaluation settings. Further, we provide a novel correction for RTM bias which
can reduce bias and attain appropriate type I error rates. This correction can
be used to perform a sensitivity analysis which determines how results may be
affected by RTM. We use our proposed correction and sensitivity analysis to
reanalyze data concerning the effects of California's Proposition 99, a
large-scale tobacco control program, on statewide smoking rates.

Methodology;Econometrics