IDEAS home Printed from https://ideas.repec.org/a/sae/medema/v42y2022i5p684-703.html
   My bibliography  Save this article

The Use of Expert Elicitation among Computational Modeling Studies in Health Research: A Systematic Review

Author

Listed:
  • Christopher J. Cadham

    (Department of Health Management and Policy, University of Michigan, School of Public Health, Ann Arbor, MI, USA)

  • Marie Knoll

    (Georgetown University, Lombardi Comprehensive Cancer Center, Washington, DC, USA)

  • Luz María Sánchez-Romero

    (Georgetown University, Lombardi Comprehensive Cancer Center, Washington, DC, USA)

  • K. Michael Cummings

    (Department of Psychiatry & Behavioral Sciences, Medical University of South Carolina, Charleston, SC, USA)

  • Clifford E. Douglas

    (Department of Health Management and Policy, University of Michigan, School of Public Health, Ann Arbor, MI, USA
    University of Michigan, Tobacco Research Network, Ann Arbor, MI, USA)

  • Alex Liber

    (Georgetown University, Lombardi Comprehensive Cancer Center, Washington, DC, USA)

  • David Mendez

    (Department of Health Management and Policy, University of Michigan, School of Public Health, Ann Arbor, MI, USA)

  • Rafael Meza

    (Department of Epidemiology, University of Michigan School of Public Health, Ann Arbor, MI, USA)

  • Ritesh Mistry

    (Department of Health Behavior and Health Education, University of Michigan School of Public Health, Ann Arbor, MI, USA)

  • Aylin Sertkaya

    (Eastern Research Group, Inc., Lexington, MA, USA)

  • Nargiz Travis

    (Department of Health Management and Policy, University of Michigan, School of Public Health, Ann Arbor, MI, USA
    Georgetown University, Lombardi Comprehensive Cancer Center, Washington, DC, USA)

  • David T. Levy

    (Georgetown University, Lombardi Comprehensive Cancer Center, Washington, DC, USA)

Abstract

Background Expert elicitation (EE) has been used across disciplines to estimate input parameters for computational modeling research when information is sparse or conflictual. Objectives We conducted a systematic review to compare EE methods used to generate model input parameters in health research. Data Sources PubMed and Web of Science. Study Eligibility Modeling studies that reported the use of EE as the source for model input probabilities were included if they were published in English before June 2021 and reported health outcomes. Data Abstraction and Synthesis Studies were classified as “formal†EE methods if they explicitly reported details of their elicitation process. Those that stated use of expert opinion but provided limited information were classified as “indeterminate†methods. In both groups, we abstracted citation details, study design, modeling methodology, a description of elicited parameters, and elicitation methods. Comparisons were made between elicitation methods. Study Appraisal Studies that conducted a formal EE were appraised on the reporting quality of the EE. Quality appraisal was not conducted for studies of indeterminate methods. Results The search identified 1520 articles, of which 152 were included. Of the included studies, 40 were classified as formal EE and 112 as indeterminate methods. Most studies were cost-effectiveness analyses (77.6%). Forty-seven indeterminate method studies provided no information on methods for generating estimates. Among formal EEs, the average reporting quality score was 9 out of 16. Limitations Elicitations on nonhealth topics and those reported in the gray literature were not included. Conclusions We found poor reporting of EE methods used in modeling studies, making it difficult to discern meaningful differences in approaches. Improved quality standards for EEs would improve the validity and replicability of computational models. Highlights We find extensive use of expert elicitation for the development of model input parameters, but most studies do not provide adequate details of their elicitation methods. Lack of reporting hinders greater discussion of the merits and challenges of using expert elicitation for model input parameter development. There is a need to establish expert elicitation best practices and reporting guidelines.

Suggested Citation

  • Christopher J. Cadham & Marie Knoll & Luz María Sánchez-Romero & K. Michael Cummings & Clifford E. Douglas & Alex Liber & David Mendez & Rafael Meza & Ritesh Mistry & Aylin Sertkaya & Nargiz Travis , 2022. "The Use of Expert Elicitation among Computational Modeling Studies in Health Research: A Systematic Review," Medical Decision Making, , vol. 42(5), pages 684-703, July.
    • Handle: RePEc:sae:medema:v:42:y:2022:i:5:p:684-703
    DOI: 10.1177/0272989X211053794
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0272989X211053794
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0272989X211053794?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Yoel Lubell & Sarah G Staedke & Brian M Greenwood & Moses R Kamya & Malcolm Molyneux & Paul N Newton & Hugh Reyburn & Robert W Snow & Umberto D'Alessandro & Mike English & Nick Day & Peter Kremsner & , 2011. "Likely Health Outcomes for Untreated Acute Febrile Illness in the Tropics in Decision and Economic Models; A Delphi Survey," PLOS ONE, Public Library of Science, vol. 6(2), pages 1-9, February.
    2. Antoine Poncet & Baris Gencer & Marc Blondon & Marianne Gex-Fabry & Christophe Combescure & Dipen Shah & Peter J Schwartz & Marie Besson & François R Girardin, 2015. "Electrocardiographic Screening for Prolonged QT Interval to Reduce Sudden Cardiac Death in Psychiatric Patients: A Cost-Effectiveness Analysis," PLOS ONE, Public Library of Science, vol. 10(6), pages 1-14, June.
    3. John Paul Gosling, 2018. "SHELF: The Sheffield Elicitation Framework," International Series in Operations Research & Management Science, in: Luis C. Dias & Alec Morton & John Quigley (ed.), Elicitation, chapter 0, pages 61-93, Springer.
    4. Matt D. Stevenson & Jeremy E. Oakley & Myfawny Lloyd Jones & Alan Brennan & Juliet E. Compston & Eugene V. McCloskey & Peter L. Selby, 2009. "The Cost-Effectiveness of an RCT to Establish Whether 5 or 10 Years of Bisphosphonate Treatment Is the Better Duration for Women With a Prior Fracture," Medical Decision Making, , vol. 29(6), pages 678-689, November.
    5. Edidiong Ekaette & Robert C. Lee & David L. Cooke & Sandra Iftody & Peter Craighead, 2007. "Probabilistic Fault Tree Analysis of a Radiation Treatment System," Risk Analysis, John Wiley & Sons, vol. 27(6), pages 1395-1410, December.
    6. Richardson, J., 1994. "Cost utility analysis: What should be measured?," Social Science & Medicine, Elsevier, vol. 39(1), pages 7-21, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Claire Copeland & Britta Turner & Gareth Powells & Kevin Wilson, 2022. "In Search of Complementarity: Insights from an Exercise in Quantifying Qualitative Energy Futures," Energies, MDPI, vol. 15(15), pages 1-21, July.
    2. Naybour, Matthew & Remenyte-Prescott, Rasa & Boyd, Matthew J., 2019. "Reliability and efficiency evaluation of a community pharmacy dispensing process using a coloured Petri-net approach," Reliability Engineering and System Safety, Elsevier, vol. 182(C), pages 258-268.
    3. Han Bleichrodt, 2002. "A new explanation for the difference between time trade‐off utilities and standard gamble utilities," Health Economics, John Wiley & Sons, Ltd., vol. 11(5), pages 447-456, July.
    4. Pope, Robin, 2004. "Biases from omitted risk effects in standard gamble utilities," Journal of Health Economics, Elsevier, vol. 23(4), pages 695-735, July.
    5. Terry Flynn, 2010. "Using Conjoint Analysis and Choice Experiments to Estimate QALY Values," PharmacoEconomics, Springer, vol. 28(9), pages 711-722, September.
    6. Duncan Mortimer & Leonie Segal, 2008. "Comparing the Incomparable? A Systematic Review of Competing Techniques for Converting Descriptive Measures of Health Status into QALY-Weights," Medical Decision Making, , vol. 28(1), pages 66-89, January.
    7. Paul F M Krabbe, 2013. "A Generalized Measurement Model to Quantify Health: The Multi-Attribute Preference Response Model," PLOS ONE, Public Library of Science, vol. 8(11), pages 1-12, November.
    8. José Luis Pinto, 1995. "Is the person trade-off a valid method for allocating health care resources? Some caveats," Economics Working Papers 140, Department of Economics and Business, Universitat Pompeu Fabra.
    9. Vo, Thang T. & Van, Pham Hoang, 2019. "Can health insurance reduce household vulnerability? Evidence from Viet Nam," World Development, Elsevier, vol. 124(C), pages 1-1.
    10. Andrea Klonschinski, 2021. "Universal Metrics for Climate Change Adaptation Finance? A Cautionary Tale," Sustainability, MDPI, vol. 13(16), pages 1-14, August.
    11. Cameron J. Williams & Kevin J. Wilson & Nina Wilson, 2021. "A comparison of prior elicitation aggregation using the classical method and SHELF," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(3), pages 920-940, July.
    12. Bleichrodt, Han & Pinto, Jose Luis & Maria Abellan-Perpinan, Jose, 2003. "A consistency test of the time trade-off," Journal of Health Economics, Elsevier, vol. 22(6), pages 1037-1052, November.
    13. Colin Green, 2001. "On the societal value of health care: what do we know about the person trade‐off technique?," Health Economics, John Wiley & Sons, Ltd., vol. 10(3), pages 233-243, April.
    14. Erik Nord & Paul Menzel & Jeff Richardson, 2006. "Multi‐method approach to valuing health states: problems with meaning," Health Economics, John Wiley & Sons, Ltd., vol. 15(2), pages 215-218, February.
    15. Christoph Werner & Tim Bedford & John Quigley, 2018. "Sequential Refined Partitioning for Probabilistic Dependence Assessment," Risk Analysis, John Wiley & Sons, vol. 38(12), pages 2683-2702, December.
    16. Aung, Yu Nandar & Tun, Sai Thein Than & Vanisaveth, Viengxay & Chindavongsa, Keobouphaphone & Kanya, Lucy, 2022. "Cost-effectiveness analysis of G6PD diagnostic test for Plasmodium vivax radical cure in Lao PDR: an economic modelling study," LSE Research Online Documents on Economics 115102, London School of Economics and Political Science, LSE Library.
    17. Chen Li & Zhihua Li & Peter Wakker, 2014. "If nudge cannot be applied: a litmus test of the readers’ stance on paternalism," Theory and Decision, Springer, vol. 76(3), pages 297-315, March.
    18. Jeroen P. Jansen & Thomas A. Trikalinos & Kathryn A. Phillips, 2022. "Assessments of the Value of New Interventions Should Include Health Equity Impact," PharmacoEconomics, Springer, vol. 40(5), pages 489-495, May.
    19. Joshua A. Salomon & Christopher J.L. Murray, 2004. "A multi‐method approach to measuring health‐state valuations," Health Economics, John Wiley & Sons, Ltd., vol. 13(3), pages 281-290, March.
    20. Dhruv Pandya & Luca Podofillini & Frank Emert & Antony J Lomax & Vinh N Dang, 2018. "Developing the foundations of a cognition-based human reliability analysis model via mapping task types and performance-influencing factors: Application to radiotherapy," Journal of Risk and Reliability, , vol. 232(1), pages 3-37, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:medema:v:42:y:2022:i:5:p:684-703. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.