IDEAS home Printed from https://ideas.repec.org/a/wly/hlthec/v22y2013i4p486-500.html
   My bibliography  Save this article

Statistical Methods For Cost‐Effectiveness Analyses That Use Observational Data: A Critical Appraisal Tool And Review Of Current Practice

Author

Listed:
  • Noémi Kreif
  • Richard Grieve
  • M. Zia Sadique

Abstract

Many cost‐effectiveness analyses (CEAs) use data from observational studies. Statistical methods can only address selection bias if they make plausible assumptions. No quality assessment tool is available for appraising CEAs that use observational studies. We developed a new checklist to assess statistical methods for addressing selection bias in CEAs that use observational data. The checklist criteria were informed by a conceptual review and applied in a systematic review of economic evaluations. Criteria included whether the study assessed the ‘no unobserved confounding’ assumption, overlap of baseline covariates between the treatment groups and the specification of the regression models. The checklist also considered structural uncertainty from the choice of statistical approach. We found 81 studies that met the inclusion criteria: studies tended to use regression (51%), matching on individual covariates (25%) or matching on the propensity score (22%). Most studies (77%) did not assess the ‘no observed confounding’ assumption, and few studies (16%) fully considered structural uncertainty from the choice of statistical approach. We conclude that published CEAs do not assess the main assumptions behind statistical methods for addressing selection bias. This checklist can raise awareness about the assumptions behind statistical methods for addressing selection bias and can complement existing method guidelines for CEAs. Copyright © 2012 John Wiley & Sons, Ltd.

Suggested Citation

  • Noémi Kreif & Richard Grieve & M. Zia Sadique, 2013. "Statistical Methods For Cost‐Effectiveness Analyses That Use Observational Data: A Critical Appraisal Tool And Review Of Current Practice," Health Economics, John Wiley & Sons, Ltd., vol. 22(4), pages 486-500, April.
  • Handle: RePEc:wly:hlthec:v:22:y:2013:i:4:p:486-500
    DOI: 10.1002/hec.2806
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/hec.2806
    Download Restriction: no

    File URL: https://libkey.io/10.1002/hec.2806?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Richard K. Crump & V. Joseph Hotz & Guido W. Imbens & Oscar A. Mitnik, 2009. "Dealing with limited overlap in estimation of average treatment effects," Biometrika, Biometrika Trust, vol. 96(1), pages 187-199.
    2. Drummond, Michael F. & Sculpher, Mark J. & Torrance, George W. & O'Brien, Bernie J. & Stoddart, Greg L., 2005. "Methods for the Economic Evaluation of Health Care Programmes," OUP Catalogue, Oxford University Press, edition 3, number 9780198529453.
    3. Andrew R. Willan & Andrew H. Briggs & Jeffrey S. Hoch, 2004. "Regression methods for covariate adjustment and subgroup analysis for non‐censored cost‐effectiveness data," Health Economics, John Wiley & Sons, Ltd., vol. 13(5), pages 461-475, May.
    4. Jeffrey S. Hoch & Andrew H. Briggs & Andrew R. Willan, 2002. "Something old, something new, something borrowed, something blue: a framework for the marriage of health econometrics and cost‐effectiveness analysis," Health Economics, John Wiley & Sons, Ltd., vol. 11(5), pages 415-430, July.
    5. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    6. Nuala A Sheehan & Vanessa Didelez & Paul R Burton & Martin D Tobin, 2008. "Mendelian Randomisation and Causal Inference in Observational Epidemiology," PLOS Medicine, Public Library of Science, vol. 5(8), pages 1-6, August.
    7. Anirban Basu & Willard G. Manning & John Mullahy, 2004. "Comparing alternative models: log vs Cox proportional hazard?," Health Economics, John Wiley & Sons, Ltd., vol. 13(8), pages 749-765, August.
    8. Joffe, M. & Mindell, J., 2006. "Complex causal process diagrams for analyzing the health impacts of policy interventions," American Journal of Public Health, American Public Health Association, vol. 96(3), pages 473-479.
    9. Thompson, Simon G. & Nixon, Richard M. & Grieve, Richard, 2006. "Addressing the issues that arise in analysing multicentre cost data, with application to a multinational study," Journal of Health Economics, Elsevier, vol. 25(6), pages 1015-1028, November.
    10. Anirban Basu & James J. Heckman & Salvador Navarro-Lozano & Sergio Urzua, 2007. "Use of instrumental variables in the presence of heterogeneity and self-selection: an application to treatments of breast cancer patients," Health Economics, John Wiley & Sons, Ltd., vol. 16(11), pages 1133-1157.
    11. Jones, A.M, 2010. "Models For Health Care," Health, Econometrics and Data Group (HEDG) Working Papers 10/01, HEDG, c/o Department of Economics, University of York.
    12. Harold C Sox & Mark Helfand & Jeremy Grimshaw & Kay Dickersin & the PLoS Medicine Editors & David Tovey & J André Knottnerus & Peter Tugwell, 2010. "Comparative Effectiveness Research: Challenges for Medical Journals," PLOS Medicine, Public Library of Science, vol. 7(4), pages 1-2, April.
    13. Steven C. Hill & G. Edward Miller, 2010. "Health expenditure estimation and functional form: applications of the generalized gamma and extended estimating equations models," Health Economics, John Wiley & Sons, Ltd., vol. 19(5), pages 608-627, May.
    14. John Mullahy, 2011. "Symposium on genetic data in health economics research," Health Economics, John Wiley & Sons, Ltd., vol. 20(8), pages 883-883, August.
    15. Ho, Daniel E. & Imai, Kosuke & King, Gary & Stuart, Elizabeth A., 2007. "Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference," Political Analysis, Cambridge University Press, vol. 15(3), pages 199-236, July.
    16. Glick, Henry A & Doshi, Jalpa A & Sonnad, Seema S & Polsky, Daniel, 2007. "Economic Evaluation in Clinical Trials," OUP Catalogue, Oxford University Press, number 9780198529972.
    17. Andrew M. Jones, 2007. "Identification of treatment effects in Health Economics," Health Economics, John Wiley & Sons, Ltd., vol. 16(11), pages 1127-1131.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Noemi Kreif & Richard Grieve & Rosalba Radice & Zia Sadique & Roland Ramsahai & Jasjeet S. Sekhon, 2012. "Methods for Estimating Subgroup Effects in Cost-Effectiveness Analyses That Use Observational Data," Medical Decision Making, , vol. 32(6), pages 750-763, November.
    2. Deidda, Manuela & Geue, Claudia & Kreif, Noemi & Dundas, Ruth & McIntosh, Emma, 2019. "A framework for conducting economic evaluations alongside natural experiments," Social Science & Medicine, Elsevier, vol. 220(C), pages 353-361.
    3. Elsa Bouée-Benhamiche & Philippe Jean Bousquet & Salah Ghabri, 2020. "Economic Evaluations of Anticancer Drugs Based on Medico-Administrative Databases: A Systematic Literature Review," Applied Health Economics and Health Policy, Springer, vol. 18(4), pages 491-508, August.
    4. Padraig Dixon & William Hollingworth & Jonathan Benger & James Calvert & Melanie Chalder & Anna King & Stephanie MacNeill & Katherine Morton & Emily Sanderson & Sarah Purdy, 2020. "Observational Cost-Effectiveness Analysis Using Routine Data: Admission and Discharge Care Bundles for Patients with Chronic Obstructive Pulmonary Disease," PharmacoEconomics - Open, Springer, vol. 4(4), pages 657-667, December.
    5. Carmen Selva-Sevilla & F Dámaso Fernández-Ginés & Manuel Cortiñas-Sáenz & Manuel Gerónimo-Pardo, 2021. "Cost-effectiveness analysis of domiciliary topical sevoflurane for painful leg ulcers," PLOS ONE, Public Library of Science, vol. 16(9), pages 1-18, September.
    6. Jason R. Guertin & Blanchard Conombo & Raphaël Langevin & Frédéric Bergeron & Anne Holbrook & Brittany Humphries & Alexis Matteau & Brian J. Potter & Christel Renoux & Jean-Éric Tarride & Madelein, 2020. "A Systematic Review of Methods Used for Confounding Adjustment in Observational Economic Evaluations in Cardiology Conducted between 2013 and 2017," Medical Decision Making, , vol. 40(5), pages 582-595, July.
    7. Pedro Saramago & Karl Claxton & Nicky J. Welton & Marta Soares, 2020. "Bayesian econometric modelling of observational data for cost‐effectiveness analysis: establishing the value of negative pressure wound therapy in the healing of open surgical wounds," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 183(4), pages 1575-1593, October.
    8. Chris Schilling & Dennis Petrie & Michelle M. Dowsey & Peter F. Choong & Philip Clarke, 2017. "The Impact of Regression to the Mean on Economic Evaluation in Quasi‐Experimental Pre–Post Studies: The Example of Total Knee Replacement Using Data from the Osteoarthritis Initiative," Health Economics, John Wiley & Sons, Ltd., vol. 26(12), pages 35-51, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jasjeet Singh Sekhon & Richard D. Grieve, 2012. "A matching method for improving covariate balance in cost‐effectiveness analyses," Health Economics, John Wiley & Sons, Ltd., vol. 21(6), pages 695-714, June.
    2. Gomes, M & Grieve, R, 2011. "Estimating the Effects of Friendship Networks on Health Behaviors of Adolescents," Health, Econometrics and Data Group (HEDG) Working Papers 11/14, HEDG, c/o Department of Economics, University of York.
    3. Manuel Gomes & Richard Grieve & Richard Nixon & W. J. Edmunds, 2012. "Statistical Methods for Cost-Effectiveness Analyses That Use Data from Cluster Randomized Trials," Medical Decision Making, , vol. 32(1), pages 209-220, January.
    4. Manuel Gomes & Richard Grieve & Richard Nixon & Edmond S.‐W. Ng & James Carpenter & Simon G. Thompson, 2012. "Methods For Covariate Adjustment In Cost‐Effectiveness Analysis That Use Cluster Randomised Trials," Health Economics, John Wiley & Sons, Ltd., vol. 21(9), pages 1101-1118, September.
    5. Li Liang & Greene Tom, 2013. "A Weighting Analogue to Pair Matching in Propensity Score Analysis," The International Journal of Biostatistics, De Gruyter, vol. 9(2), pages 215-234, July.
    6. Jeffrey Smith & Arthur Sweetman, 2016. "Viewpoint: Estimating the causal effects of policies and programs," Canadian Journal of Economics, Canadian Economics Association, vol. 49(3), pages 871-905, August.
    7. Ferraro, Paul J. & Miranda, Juan José, 2014. "The performance of non-experimental designs in the evaluation of environmental programs: A design-replication study using a large-scale randomized experiment as a benchmark," Journal of Economic Behavior & Organization, Elsevier, vol. 107(PA), pages 344-365.
    8. Miquel-Àngel Garcia-López & Albert Solé-Ollé & Elisabet Viladecans Marsal, 2014. "Do Land Use Policies Follow Road Construction," CESifo Working Paper Series 4672, CESifo Group Munich.
    9. Martin Huber & Michael Lechner & Andreas Steinmayr, 2015. "Radius matching on the propensity score with bias adjustment: tuning parameters and finite sample behaviour," Empirical Economics, Springer, vol. 49(1), pages 1-31, August.
    10. Paul Ferraro & Merlin Hanauer, 2011. "Protecting Ecosystems and Alleviating Poverty with Parks and Reserves: ‘Win-Win’ or Tradeoffs?," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 48(2), pages 269-286, February.
    11. McCarthy, Ian M., 2016. "Eliminating composite bias in treatment effects estimates: Applications to quality of life assessment," Journal of Health Economics, Elsevier, vol. 50(C), pages 47-58.
    12. Daisuke Goto & Ya-Chen Tina Shih & Pascal Lecomte & Melvin Olson & Chukwukadibia Udeze & Yujin Park & C. Daniel Mullins, 2017. "Regression-Based Approaches to Patient-Centered Cost-Effectiveness Analysis," PharmacoEconomics, Springer, vol. 35(7), pages 685-695, July.
    13. Huber, Martin & Lechner, Michael & Wunsch, Conny, 2013. "The performance of estimators based on the propensity score," Journal of Econometrics, Elsevier, vol. 175(1), pages 1-21.
    14. Jones, A. & Lomas, J. & Rice, N., 2014. "Going Beyond the Mean in Healthcare Cost Regressions: a Comparison of Methods for Estimating the Full Conditional Distribution," Health, Econometrics and Data Group (HEDG) Working Papers 14/26, HEDG, c/o Department of Economics, University of York.
    15. Andrew M. Jones & James Lomas & Peter T. Moore & Nigel Rice, 2016. "A quasi-Monte-Carlo comparison of parametric and semiparametric regression methods for heavy-tailed and non-normal data: an application to healthcare costs," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 179(4), pages 951-974, October.
    16. Miquel-Àngel Garcia-López & Albert Solé-Ollé & Elisabet Viladecans Marsal, 2014. "Do Land Use Policies Follow Road Construction," CESifo Working Paper Series 4672, CESifo.
    17. Vladimir Atanasov & Bernard Black, 2021. "The Trouble with Instruments: The Need for Pretreatment Balance in Shock-Based Instrumental Variable Designs," Management Science, INFORMS, vol. 67(2), pages 1270-1302, February.
    18. Theodoros Mantopoulos & Paul M. Mitchell & Nicky J. Welton & Richard McManus & Lazaros Andronis, 2016. "Choice of statistical model for cost-effectiveness analysis and covariate adjustment: empirical application of prominent models and assessment of their results," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 17(8), pages 927-938, November.
    19. Huber, Martin & Lechner, Michael & Wunsch, Conny, 2010. "How to Control for Many Covariates? Reliable Estimators Based on the Propensity Score," IZA Discussion Papers 5268, Institute of Labor Economics (IZA).
    20. Andrew M. Jones & James Lomas & Nigel Rice, 2015. "Healthcare Cost Regressions: Going Beyond the Mean to Estimate the Full Distribution," Health Economics, John Wiley & Sons, Ltd., vol. 24(9), pages 1192-1212, September.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:hlthec:v:22:y:2013:i:4:p:486-500. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://www3.interscience.wiley.com/cgi-bin/jhome/5749 .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www3.interscience.wiley.com/cgi-bin/jhome/5749 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.