IDEAS home Printed from https://ideas.repec.org/a/eee/socmed/v220y2019icp353-361.html
   My bibliography  Save this article

A framework for conducting economic evaluations alongside natural experiments

Author

Listed:
  • Deidda, Manuela
  • Geue, Claudia
  • Kreif, Noemi
  • Dundas, Ruth
  • McIntosh, Emma

Abstract

Internationally, policy makers are increasingly focussed on reducing the detrimental consequences and rising costs associated with unhealthy diets, inactivity, smoking, alcohol and other risk factors on the health of their populations. This has led to an increase in the demand for evidence-based, cost-effective Population Health Interventions (PHIs) to reverse this trend. Given that research designs such as randomised controlled trials (RCTs) are often not suited to the evaluation of PHIs, Natural Experiments (NEs) are now frequently being used as a design to evaluate such complex, preventive PHIs. However, current guidance for economic evaluation focusses on RCT designs and therefore does not address the specific challenges of NE designs. Using such guidance can lead to sub-optimal design, data collection and analysis for NEs, leading to bias in the estimated effectiveness and cost-effectiveness of the PHI. As a consequence, there is a growing recognition of the need to identify a robust methodological framework for the design and conducting of economic evaluations alongside such NEs. This paper outlines the challenges inherent to the design and conduct of economic evaluations of PHIs alongside NEs, providing a comprehensive framework and outlining a research agenda in this area.

Suggested Citation

  • Deidda, Manuela & Geue, Claudia & Kreif, Noemi & Dundas, Ruth & McIntosh, Emma, 2019. "A framework for conducting economic evaluations alongside natural experiments," Social Science & Medicine, Elsevier, vol. 220(C), pages 353-361.
  • Handle: RePEc:eee:socmed:v:220:y:2019:i:c:p:353-361
    DOI: 10.1016/j.socscimed.2018.11.032
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S027795361830666X
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.socscimed.2018.11.032?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Noemi Kreif & Richard Grieve & Rosalba Radice & Zia Sadique & Roland Ramsahai & Jasjeet S. Sekhon, 2012. "Methods for Estimating Subgroup Effects in Cost-Effectiveness Analyses That Use Observational Data," Medical Decision Making, , vol. 32(6), pages 750-763, November.
    2. Kalipso Chalkidou & Anthony Culyer & Bhash Naidoo & Peter Littlejohns, 2008. "Cost‐effective public health guidance: asking questions from the decision‐maker's viewpoint," Health Economics, John Wiley & Sons, Ltd., vol. 17(3), pages 441-448, March.
    3. Sebastian Calonico & Matias D. Cattaneo & Rocio Titiunik, 2014. "Robust data-driven inference in the regression-discontinuity design," Stata Journal, StataCorp LP, vol. 14(4), pages 909-946, December.
    4. Noémi Kreif & Richard Grieve & M. Zia Sadique, 2013. "Statistical Methods For Cost‐Effectiveness Analyses That Use Observational Data: A Critical Appraisal Tool And Review Of Current Practice," Health Economics, John Wiley & Sons, Ltd., vol. 22(4), pages 486-500, April.
    5. Don Husereau & Michael Drummond & Stavros Petrou & Chris Carswell & David Moher & Dan Greenberg & Federico Augustovski & Andrew Briggs & Josephine Mauskopf & Elizabeth Loder, 2013. "Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 14(3), pages 367-372, June.
    6. Mabel Andalón, 2011. "Oportunidades to reduce overweight and obesity in Mexico?," Health Economics, John Wiley & Sons, Ltd., vol. 20(S1), pages 1-18, September.
    7. Arijit Nandi & Mohammad Hajizadeh & Sam Harper & Alissa Koski & Erin C Strumpf & Jody Heymann, 2016. "Increased Duration of Paid Maternity Leave Lowers Infant Mortality in Low- and Middle-Income Countries: A Quasi-Experimental Study," PLOS Medicine, Public Library of Science, vol. 13(3), pages 1-18, March.
    8. Sarah Byford & Tom Sefton, 2003. "Economic Evaluation of Complex Health and Social Care Interventions," National Institute Economic Review, National Institute of Economic and Social Research, vol. 186(1), pages 98-108, October.
    9. Jens Ludwig & Douglas L. Miller, 2007. "Does Head Start Improve Children's Life Chances? Evidence from a Regression Discontinuity Design," The Quarterly Journal of Economics, Oxford University Press, vol. 122(1), pages 159-208.
    10. Giulia Greco & Paula Lorgelly & Inthira Yamabhai, 2016. "Outcomes in Economic Evaluations of Public Health Interventions in Low‐ and Middle‐Income Countries: Health, Capabilities and Subjective Wellbeing," Health Economics, John Wiley & Sons, Ltd., vol. 25(S1), pages 83-94, February.
    11. Terza, Joseph V. & Basu, Anirban & Rathouz, Paul J., 2008. "Two-stage residual inclusion estimation: Addressing endogeneity in health econometric modeling," Journal of Health Economics, Elsevier, vol. 27(3), pages 531-543, May.
    12. Miqdad Asaria & Susan Griffin & Richard Cookson, 2013. "Distributional cost-effectiveness analysis: a tutorial," Working Papers 092cherp, Centre for Health Economics, University of York.
    13. Marco Caliendo & Sabine Kopeinig, 2008. "Some Practical Guidance For The Implementation Of Propensity Score Matching," Journal of Economic Surveys, Wiley Blackwell, vol. 22(1), pages 31-72, February.
    14. Deaton, Angus & Cartwright, Nancy, 2018. "Understanding and misunderstanding randomized controlled trials," Social Science & Medicine, Elsevier, vol. 210(C), pages 2-21.
    15. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    16. Rita Faria & Manuel Gomes & David Epstein & Ian White, 2014. "A Guide to Handling Missing Data in Cost-Effectiveness Analysis Conducted Within Randomised Controlled Trials," PharmacoEconomics, Springer, vol. 32(12), pages 1157-1170, December.
    17. Manuel Gomes & Edmond S.-W. Ng & Richard Grieve & Richard Nixon & James Carpenter & Simon G. Thompson, 2012. "Developing Appropriate Methods for Cost-Effectiveness Analysis of Cluster Randomized Trials," Medical Decision Making, , vol. 32(2), pages 350-361, March.
    18. Meyer, Bruce D, 1995. "Natural and Quasi-experiments in Economics," Journal of Business & Economic Statistics, American Statistical Association, vol. 13(2), pages 151-161, April.
    19. Steven T. Yen & Margaret Andrews & Zhuo Chen & David B. Eastwood, 2008. "Food Stamp Program Participation and Food Insecurity: An Instrumental Variables Approach," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 90(1), pages 117-132.
    20. Imbens, Guido W. & Lemieux, Thomas, 2008. "Regression discontinuity designs: A guide to practice," Journal of Econometrics, Elsevier, vol. 142(2), pages 615-635, February.
    21. Des Jarlais, D.C. & Lyles, C. & Crepaz, N., 2004. "Improving the Reporting Quality of Nonrandomized Evaluations of Behavioral and Public Health Interventions: The TREND Statement," American Journal of Public Health, American Public Health Association, vol. 94(3), pages 361-366.
    22. Susan Athey & Guido W. Imbens, 2017. "The State of Applied Econometrics: Causality and Policy Evaluation," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 3-32, Spring.
    23. Briggs, Andrew & Sculpher, Mark & Claxton, Karl, 2006. "Decision Modelling for Health Economic Evaluation," OUP Catalogue, Oxford University Press, number 9780198526629.
    24. Jasjeet Singh Sekhon & Richard D. Grieve, 2012. "A matching method for improving covariate balance in cost‐effectiveness analyses," Health Economics, John Wiley & Sons, Ltd., vol. 21(6), pages 695-714, June.
    25. K. DiazOrdaz & A. J. Franchini & R. Grieve, 2018. "Methods for estimating complier average causal effects for cost‐effectiveness analysis," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 181(1), pages 277-297, January.
    26. Stéphane Verguet & Ramanan Laxminarayan & Dean T. Jamison, 2015. "Universal Public Finance of Tuberculosis Treatment in India: An Extended Cost‐Effectiveness Analysis," Health Economics, John Wiley & Sons, Ltd., vol. 24(3), pages 318-332, March.
    27. Wagenaar, A.C. & O'Malley, P.M. & LaFond, C., 2001. "Lowered legal blood alcohol limits for young drivers: Effects on drinking, driving, and driving-after-drinking behaviors in 30 states," American Journal of Public Health, American Public Health Association, vol. 91(5), pages 801-804.
    28. Giulia Greco & Paula Lorgelly & Inthira Yamabhai, 2016. "Outcomes in Economic Evaluations of Public Health Interventions in Low‐ and Middle‐Income Countries: Health, Capabilities and Subjective Wellbeing," Health Economics, John Wiley & Sons, Ltd., vol. 25, pages 83-94, February.
    29. Weatherly, Helen & Drummond, Michael & Claxton, Karl & Cookson, Richard & Ferguson, Brian & Godfrey, Christine & Rice, Nigel & Sculpher, Mark & Sowden, Amanda, 2009. "Methods for assessing the cost-effectiveness of public health interventions: Key challenges and recommendations," Health Policy, Elsevier, vol. 93(2-3), pages 85-92, December.
    30. Abadie, Alberto & Diamond, Alexis & Hainmueller, Jens, 2010. "Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program," Journal of the American Statistical Association, American Statistical Association, vol. 105(490), pages 493-505.
    31. Merkur, Sherry & Sassi, Franco & McDaid, David, 2013. "Promoting health, preventing disease: is there an economic case?," LSE Research Online Documents on Economics 55659, London School of Economics and Political Science, LSE Library.
    32. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881, October.
    33. Glick, Henry A & Doshi, Jalpa A & Sonnad, Seema S & Polsky, Daniel, 2007. "Economic Evaluation in Clinical Trials," OUP Catalogue, Oxford University Press, number 9780198529972.
    34. Drummond, Michael F. & Sculpher, Mark J. & Claxton, Karl & Stoddart, Greg L. & Torrance, George W., 2015. "Methods for the Economic Evaluation of Health Care Programmes," OUP Catalogue, Oxford University Press, edition 4, number 9780199665884.
    35. Ichida, Yukinobu & Hirai, Hiroshi & Kondo, Katsunori & Kawachi, Ichiro & Takeda, Tokunori & Endo, Hideki, 2013. "Does social participation improve self-rated health in the older population? A quasi-experimental intervention study," Social Science & Medicine, Elsevier, vol. 94(C), pages 83-90.
    36. Anirban Basu & Andrea Manca, 2012. "Regression Estimators for Generic Health-Related Quality of Life and Quality-Adjusted Life Years," Medical Decision Making, , vol. 32(1), pages 56-69, January.
    37. Richard M. Nixon & Simon G. Thompson, 2005. "Methods for incorporating covariate adjustment, subgroup analysis and between‐centre differences into cost‐effectiveness evaluations," Health Economics, John Wiley & Sons, Ltd., vol. 14(12), pages 1217-1229, December.
    38. Dimitrios Rovithis, 2013. "Do health economic evaluations using observational data provide reliable assessment of treatment effects?," Health Economics Review, Springer, vol. 3(1), pages 1-7, December.
    39. José R. Zubizarreta, 2012. "Using Mixed Integer Programming for Matching in an Observational Study of Kidney Failure After Surgery," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 107(500), pages 1360-1371, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jason R. Guertin & Blanchard Conombo & Raphaël Langevin & Frédéric Bergeron & Anne Holbrook & Brittany Humphries & Alexis Matteau & Brian J. Potter & Christel Renoux & Jean-Éric Tarride & Madelein, 2020. "A Systematic Review of Methods Used for Confounding Adjustment in Observational Economic Evaluations in Cardiology Conducted between 2013 and 2017," Medical Decision Making, , vol. 40(5), pages 582-595, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Andrew Briggs & Rachel Nugent, 2016. "Editorial," Health Economics, John Wiley & Sons, Ltd., vol. 25(S1), pages 6-8, February.
    2. Peter Ganong & Simon Jäger, 2018. "A Permutation Test for the Regression Kink Design," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(522), pages 494-504, April.
    3. Hill, Sarah R. & Vale, Luke & Hunter, David & Henderson, Emily & Oluboyede, Yemi, 2017. "Economic evaluations of alcohol prevention interventions: Is the evidence sufficient? A review of methodological challenges," Health Policy, Elsevier, vol. 121(12), pages 1249-1262.
    4. Bruno Ferman & Cristine Pinto & Vitor Possebom, 2020. "Cherry Picking with Synthetic Controls," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 39(2), pages 510-532, March.
    5. Dennis Shen & Peng Ding & Jasjeet Sekhon & Bin Yu, 2022. "Same Root Different Leaves: Time Series and Cross-Sectional Methods in Panel Data," Papers 2207.14481, arXiv.org, revised Oct 2022.
    6. Denis Fougère & Nicolas Jacquemet, 2020. "Policy Evaluation Using Causal Inference Methods," SciencePo Working papers Main hal-03455978, HAL.
    7. Guido W. Imbens, 2020. "Potential Outcome and Directed Acyclic Graph Approaches to Causality: Relevance for Empirical Practice in Economics," Journal of Economic Literature, American Economic Association, vol. 58(4), pages 1129-1179, December.
    8. Giulio Grossi & Marco Mariani & Alessandra Mattei & Patrizia Lattarulo & Ozge Oner, 2020. "Direct and spillover effects of a new tramway line on the commercial vitality of peripheral streets. A synthetic-control approach," Papers 2004.05027, arXiv.org, revised Nov 2023.
    9. Nicolas R. Ziebarth, 2018. "Social Insurance and Health," Contributions to Economic Analysis, in: Health Econometrics, volume 127, pages 57-84, Emerald Group Publishing Limited.
    10. Kraft, Kornelius & Lammers, Alexander, 2021. "Bargaining Power and the Labor Share - a Structural Break Approach," VfS Annual Conference 2021 (Virtual Conference): Climate Economics 242342, Verein für Socialpolitik / German Economic Association.
    11. Susan Athey & Guido W. Imbens, 2017. "The State of Applied Econometrics: Causality and Policy Evaluation," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 3-32, Spring.
    12. Díaz, Juan & Grau, Nicolás & Reyes, Tatiana & Rivera, Jorge, 2021. "The impact of grade retention on juvenile crime," Economics of Education Review, Elsevier, vol. 84(C).
    13. Carmen Selva-Sevilla & Elena Conde-Montero & Manuel Gerónimo-Pardo, 2020. "Bayesian Regression Model for a Cost-Utility and Cost-Effectiveness Analysis Comparing Punch Grafting Versus Usual Care for the Treatment of Chronic Wounds," IJERPH, MDPI, vol. 17(11), pages 1-21, May.
    14. Brett R. Gordon & Florian Zettelmeyer & Neha Bhargava & Dan Chapsky, 2019. "A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook," Marketing Science, INFORMS, vol. 38(2), pages 193-225, March.
    15. Boris Salazar-Trujillo & Daniel Otero Robles, 2019. "La revolución empírica en economía," Apuntes del Cenes, Universidad Pedagógica y Tecnológica de Colombia, vol. 38(68), pages 15-48, July.
    16. Carmen Selva-Sevilla & F Dámaso Fernández-Ginés & Manuel Cortiñas-Sáenz & Manuel Gerónimo-Pardo, 2021. "Cost-effectiveness analysis of domiciliary topical sevoflurane for painful leg ulcers," PLOS ONE, Public Library of Science, vol. 16(9), pages 1-18, September.
    17. Alexina J. Mason & Manuel Gomes & James Carpenter & Richard Grieve, 2021. "Flexible Bayesian longitudinal models for cost‐effectiveness analyses with informative missing data," Health Economics, John Wiley & Sons, Ltd., vol. 30(12), pages 3138-3158, December.
    18. Matthew A. Masten & Alexandre Poirier & Linqi Zhang, 2020. "Assessing Sensitivity to Unconfoundedness: Estimation and Inference," Papers 2012.15716, arXiv.org.
    19. Mohamed El Alili & Johanna M. van Dongen & Jonas L. Esser & Martijn W. Heymans & Maurits W. van Tulder & Judith E. Bosmans, 2022. "A scoping review of statistical methods for trial‐based economic evaluations: The current state of play," Health Economics, John Wiley & Sons, Ltd., vol. 31(12), pages 2680-2699, December.
    20. Fattore, Giovanni & Federici, Carlo & Drummond, Michael & Mazzocchi, Mario & Detzel, Patrick & Hutton, Zsuzsa V & Shankar, Bhavani, 2021. "Economic evaluation of nutrition interventions: Does one size fit all?," Health Policy, Elsevier, vol. 125(9), pages 1238-1246.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:socmed:v:220:y:2019:i:c:p:353-361. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/315/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.