IDEAS home Printed from https://ideas.repec.org/a/wly/hlthec/v25y2016i12p1514-1528.html
   My bibliography  Save this article

Examination of the Synthetic Control Method for Evaluating Health Policies with Multiple Treated Units

Author

Listed:
  • Noémi Kreif
  • Richard Grieve
  • Dominik Hangartner
  • Alex James Turner
  • Silviya Nikolova
  • Matt Sutton

Abstract

This paper examines the synthetic control method in contrast to commonly used difference-in-differences (DiD) estimation, in the context of a re-evaluation of a pay-for-performance (P4P) initiative, the Advancing Quality scheme. The synthetic control method aims to estimate treatment effects by constructing a weighted combination of control units, which represents what the treated group would have experienced in the absence of receiving the treatment. While DiD estimation assumes that the effects of unobserved confounders are constant over time, the synthetic control method allows for these effects to change over time, by re-weighting the control group so that it has similar pre-intervention characteristics to the treated group. We extend the synthetic control approach to a setting of evaluation of a health policy where there are multiple treated units. We re-analyse a recent study evaluating the effects of a hospital P4P scheme on risk-adjusted hospital mortality. In contrast to the original DiD analysis, the synthetic control method reports that, for the incentivised conditions, the P4P scheme did not significantly reduce mortality and that there is a statistically significant increase in mortality for non-incentivised conditions. This result was robust to alternative specifications of the synthetic control method
(This abstract was borrowed from another version of this item.)

Suggested Citation

  • Noémi Kreif & Richard Grieve & Dominik Hangartner & Alex James Turner & Silviya Nikolova & Matt Sutton, 2016. "Examination of the Synthetic Control Method for Evaluating Health Policies with Multiple Treated Units," Health Economics, John Wiley & Sons, Ltd., vol. 25(12), pages 1514-1528, December.
  • Handle: RePEc:wly:hlthec:v:25:y:2016:i:12:p:1514-1528
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. Richard Blundell & Monica Costa Dias, 2009. "Alternative Approaches to Evaluation in Empirical Microeconomics," Journal of Human Resources, University of Wisconsin Press, vol. 44(3).
    2. Coffman, Makena & Noy, Ilan, 2012. "Hurricane Iniki: measuring the long-term economic impact of a natural disaster using synthetic control," Environment and Development Economics, Cambridge University Press, vol. 17(2), pages 187-205, April.
    3. A. Colin Cameron & Jonah B. Gelbach & Douglas L. Miller, 2011. "Robust Inference With Multiway Clustering," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 29(2), pages 238-249, April.
    4. Brewer Mike & Crossley Thomas F. & Joyce Robert, 2018. "Inference with Difference-in-Differences Revisited," Journal of Econometric Methods, De Gruyter, vol. 7(1), pages 1-16, January.
    5. Rachel Meacock & Søren Rud Kristensen & Matt Sutton, 2014. "The Cost‐Effectiveness Of Using Financial Incentives To Improve Provider Quality: A Framework And Application," Health Economics, John Wiley & Sons, Ltd., vol. 23(1), pages 1-13, January.
    6. Alberto Abadie & Alexis Diamond & Jens Hainmueller, 2015. "Comparative Politics and the Synthetic Control Method," American Journal of Political Science, John Wiley & Sons, vol. 59(2), pages 495-510, February.
    7. Charles J. Courtemanche & Daniela Zapata, 2014. "Does Universal Coverage Improve Health? The Massachusetts Experience," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 33(1), pages 36-69, January.
    8. Currie, Janet & Stabile, Mark & Jones, Lauren, 2014. "Do stimulant medications improve educational and behavioral outcomes for children with ADHD?," Journal of Health Economics, Elsevier, vol. 37(C), pages 58-69.
    9. Andreas Billmeier & Tommaso Nannicini, 2013. "Assessing Economic Liberalization Episodes: A Synthetic Control Approach," The Review of Economics and Statistics, MIT Press, vol. 95(3), pages 983-1001, July.
    10. Susan Athey & Guido W. Imbens, 2006. "Identification and Inference in Nonlinear Difference-in-Differences Models," Econometrica, Econometric Society, vol. 74(2), pages 431-497, March.
    11. Martin Emmert & Frank Eijkenaar & Heike Kemter & Adelheid Esslinger & Oliver Schöffski, 2012. "Economic evaluation of pay-for-performance in health care: a systematic review," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 13(6), pages 755-767, December.
    12. Frank Eijkenaar, 2013. "Key issues in the design of pay for performance programs," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 14(1), pages 117-131, February.
    13. Marianne Bertrand & Esther Duflo & Sendhil Mullainathan, 2004. "How Much Should We Trust Differences-In-Differences Estimates?," The Quarterly Journal of Economics, Oxford University Press, vol. 119(1), pages 249-275.
    14. Hugh Gravelle & Matt Sutton & Ada Ma, 2010. "Doctor Behaviour under a Pay for Performance Contract: Treating, Cheating and Case Finding?," Economic Journal, Royal Economic Society, vol. 120(542), pages 129-156, February.
    15. Acemoglu, Daron & Johnson, Simon & Kermani, Amir & Kwak, James & Mitton, Todd, 2016. "The value of connections in turbulent times: Evidence from the United States," Journal of Financial Economics, Elsevier, vol. 121(2), pages 368-391.
    16. Alberto Abadie, 2005. "Semiparametric Difference-in-Differences Estimators," Review of Economic Studies, Oxford University Press, vol. 72(1), pages 1-19.
    17. Roland G. Fryer, 2011. "Teacher Incentives and Student Achievement: Evidence from New York City Public Schools," NBER Working Papers 16850, National Bureau of Economic Research, Inc.
    18. Peter Hinrichs, 2012. "The Effects of Affirmative Action Bans on College Enrollment, Educational Attainment, and the Demographic Composition of Universities," The Review of Economics and Statistics, MIT Press, vol. 94(3), pages 712-722, August.
    19. Abadie, Alberto & Diamond, Alexis & Hainmueller, Jens, 2010. "Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program," Journal of the American Statistical Association, American Statistical Association, vol. 105(490), pages 493-505.
    Full references (including those not matched with items on IDEAS)

    More about this item

    JEL classification:

    • N0 - Economic History - - General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:hlthec:v:25:y:2016:i:12:p:1514-1528. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Wiley Content Delivery). General contact details of provider: http://www3.interscience.wiley.com/cgi-bin/jhome/5749 .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.