IDEAS home Printed from https://ideas.repec.org/a/sae/medema/v36y2016i5p652-665.html
   My bibliography  Save this article

Calibrating Parameters for Microsimulation Disease Models

Author

Listed:
  • Alex van der Steen
  • Joost van Rosmalen
  • Sonja Kroep
  • Frank van Hees
  • Ewout W. Steyerberg
  • Harry J. de Koning
  • Marjolein van Ballegooijen
  • Iris Lansdorp-Vogelaar

Abstract

Background. Calibration (estimation of model parameters) compares model outcomes with observed outcomes and explores possible model parameter values to identify the set of values that provides the best fit to the data. The goodness-of-fit (GOF) criterion quantifies the difference between model and observed outcomes. There is no consensus on the most appropriate GOF criterion, because a direct performance comparison of GOF criteria in model calibration is lacking. Methods. We systematically compared the performance of commonly used GOF criteria (sum of squared errors [SSE], Pearson chi-square, and a likelihood-based approach [Poisson and/or binomial deviance functions]) in the calibration of selected parameters of the MISCAN-Colon microsimulation model for colorectal cancer. The performance of each GOF criterion was assessed by comparing the 1) root mean squared prediction error (RMSPE) of the selected parameters, 2) computation time of the calibration procedure of various calibration scenarios, and 3) impact on estimated cost-effectiveness ratios. Results. The likelihood-based deviance resulted in the lowest RMSPE in 4 of 6 calibration scenarios and was close to best in the other 2. The SSE had a 25 times higher RMSPE in a scenario with considerable differences in the values of observed outcomes, whereas the Pearson chi-square had a 60 times higher RMSPE in a scenario with multiple studies measuring the same outcome. In all scenarios, the SSE required the most computation time. The likelihood-based approach estimated the cost-effectiveness ratio most accurately (up to −0.15% relative difference versus 0.44% [SSE] and 13% [Pearson chi-square]). Conclusions. The likelihood-based deviance criteria lead to accurate estimation of parameters under various circumstances. These criteria are recommended for calibration in microsimulation disease models in contrast with other commonly used criteria.

Suggested Citation

  • Alex van der Steen & Joost van Rosmalen & Sonja Kroep & Frank van Hees & Ewout W. Steyerberg & Harry J. de Koning & Marjolein van Ballegooijen & Iris Lansdorp-Vogelaar, 2016. "Calibrating Parameters for Microsimulation Disease Models," Medical Decision Making, , vol. 36(5), pages 652-665, July.
    • Handle: RePEc:sae:medema:v:36:y:2016:i:5:p:652-665
    DOI: 10.1177/0272989X16636851
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0272989X16636851
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0272989X16636851?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Heij, Christiaan & de Boer, Paul & Franses, Philip Hans & Kloek, Teun & van Dijk, Herman K., 2004. "Econometric Methods with Applications in Business and Economics," OUP Catalogue, Oxford University Press, number 9780199268016.
    2. Alan Brennan & Stephen E. Chick & Ruth Davies, 2006. "A taxonomy of model structures for economic evaluation of health technologies," Health Economics, John Wiley & Sons, Ltd., vol. 15(12), pages 1295-1310, December.
    3. Neddermeijer, H.G. & van Oortmarssen, G.J. & Piersma, N. & Dekker, R. & Habbema, J.D.F., 2000. "Adaptive extensions of the Nelder and Mead Simplex Method for optimization of stochastic simulation models," Econometric Institute Research Papers EI 2000-22/A, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    4. Rutter, Carolyn M. & Miglioretti, Diana L. & Savarino, James E., 2009. "Bayesian Calibration of Microsimulation Models," Journal of the American Statistical Association, American Statistical Association, vol. 104(488), pages 1338-1350.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jing Voon Chen & Julia L. Higle & Michael Hintlian, 2018. "A systematic approach for examining the impact of calibration uncertainty in disease modeling," Computational Management Science, Springer, vol. 15(3), pages 541-561, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Franses, Ph.H.B.F. & van Oest, R.D., 2006. "Testing changes in consumer confidence indicators," Econometric Institute Research Papers EI 2006-18, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    2. Blum, Bianca, 2018. "Ausgestaltung einer Steuerpolitik zur Förderung von LED-Beleuchtung," The Constitutional Economics Network Working Papers 01-2018, University of Freiburg, Department of Economic Policy and Constitutional Economic Theory.
    3. Hassan Belkacem Ghassan & Abdelkrim Ahmed Guendouz, 2019. "Panel modeling of z-score: evidence from Islamic and conventional Saudi banks," International Journal of Islamic and Middle Eastern Finance and Management, Emerald Group Publishing Limited, vol. 12(3), pages 448-468, July.
    4. Heß, Michael (Ed.) & Schlieter, Hannes (Ed.), 2014. "Modellierung im Gesundheitswesen: Tagungsband des Workshops im Rahmen der Modellierung 2014," ICB Research Reports 57, University Duisburg-Essen, Institute for Computer Science and Business Information Systems (ICB).
    5. Niklas Elert, 2014. "What determines entry? Evidence from Sweden," The Annals of Regional Science, Springer;Western Regional Science Association, vol. 53(1), pages 55-92, August.
    6. Ivan Jericevich & Murray McKechnie & Tim Gebbie, 2021. "Calibrating an adaptive Farmer-Joshi agent-based model for financial markets," Papers 2104.09863, arXiv.org.
    7. Vaz, Thalita A. & Machado, Sérgio J. & Bortoluzzo, Adriana B., 2011. "Estimation of Conversion Rates into Annuities: A Brazilian Perspective," Insper Working Papers wpe_249, Insper Working Paper, Insper Instituto de Ensino e Pesquisa.
    8. Ortiz-Barrios, Miguel & Arias-Fonseca, Sebastián & Ishizaka, Alessio & Barbati, Maria & Avendaño-Collante, Betty & Navarro-Jiménez, Eduardo, 2023. "Artificial intelligence and discrete-event simulation for capacity management of intensive care units during the Covid-19 pandemic: A case study," Journal of Business Research, Elsevier, vol. 160(C).
    9. Odolinski, Kristofer & Wheat, Phillip, 2016. "Dynamics in rail infrastructure provision: maintenance and renewal costs in Sweden," Working papers in Transport Economics 2016:23, CTS - Centre for Transport Studies Stockholm (KTH and VTI), revised 11 Dec 2017.
    10. Bernhard Ultsch & Oliver Damm & Philippe Beutels & Joke Bilcke & Bernd Brüggenjürgen & Andreas Gerber-Grote & Wolfgang Greiner & Germaine Hanquet & Raymond Hutubessy & Mark Jit & Mirjam Knol & Rüdiger, 2016. "Methods for Health Economic Evaluation of Vaccines and Immunization Decision Frameworks: A Consensus Framework from a European Vaccine Economics Community," PharmacoEconomics, Springer, vol. 34(3), pages 227-244, March.
    11. Hossein Haji Ali Afzali & Laura Bojke & Jonathan Karnon, 2018. "Model Structuring for Economic Evaluations of New Health Technologies," PharmacoEconomics, Springer, vol. 36(11), pages 1309-1319, November.
    12. Tan, S.Y.G.L. & van Oortmarssen, G.J. & Piersma, N., 2000. "Estimting parameters of a microsimulation model for breast cancer screening using the score function method," Econometric Institute Research Papers EI 2000-35/A, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    13. Becky Pennington & Alex Filby & Lesley Owen & Matthew Taylor, 2018. "Smoking Cessation: A Comparison of Two Model Structures," PharmacoEconomics, Springer, vol. 36(9), pages 1101-1112, September.
    14. Gemma E. Shields & Mark Wilberforce & Paul Clarkson & Tracey Farragher & Arpana Verma & Linda M. Davies, 2022. "Factors Limiting Subgroup Analysis in Cost-Effectiveness Analysis and a Call for Transparency," PharmacoEconomics, Springer, vol. 40(2), pages 149-156, February.
    15. Fernando Alarid-Escudero & Richard F. MacLehose & Yadira Peralta & Karen M. Kuntz & Eva A. Enns, 2018. "Nonidentifiability in Model Calibration and Implications for Medical Decision Making," Medical Decision Making, , vol. 38(7), pages 810-821, October.
    16. Koen Degeling & Maarten J. IJzerman & Mariel S. Lavieri & Mark Strong & Hendrik Koffijberg, 2020. "Introduction to Metamodeling for Reducing Computational Burden of Advanced Analyses with Health Economic Models: A Structured Overview of Metamodeling Methods in a 6-Step Application Process," Medical Decision Making, , vol. 40(3), pages 348-363, April.
    17. Peter J. Dodd & Jeff J. Pennington & Liza Bronner Murrison & David W. Dowdy, 2018. "Simple Inclusion of Complex Diagnostic Algorithms in Infectious Disease Models for Economic Evaluation," Medical Decision Making, , vol. 38(8), pages 930-941, November.
    18. Jonathan Karnon & James Stahl & Alan Brennan & J. Jaime Caro & Javier Mar & Jörgen Möller, 2012. "Modeling Using Discrete Event Simulation," Medical Decision Making, , vol. 32(5), pages 701-711, September.
    19. Jing Voon Chen & Julia L. Higle & Michael Hintlian, 2018. "A systematic approach for examining the impact of calibration uncertainty in disease modeling," Computational Management Science, Springer, vol. 15(3), pages 541-561, October.
    20. Faruk Balli & Hassan Ghassan & Essam H. Jeefri, 2021. "Sukuk and bond spreads," Journal of Economics and Finance, Springer;Academy of Economics and Finance, vol. 45(3), pages 529-543, July.
      • Balli, Faruk & Ghassan, Hassan B. & Al-Jefri, Essam H., 2020. "Sukuk and bond spreads," MPRA Paper 106729, University Library of Munich, Germany, revised 20 Jan 2021.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:medema:v:36:y:2016:i:5:p:652-665. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.