IDEAS home Printed from https://ideas.repec.org/a/sae/medema/v36y2016i5p652-665.html

Calibrating Parameters for Microsimulation Disease Models

Author

Listed:
  • Alex van der Steen
  • Joost van Rosmalen
  • Sonja Kroep
  • Frank van Hees
  • Ewout W. Steyerberg
  • Harry J. de Koning
  • Marjolein van Ballegooijen
  • Iris Lansdorp-Vogelaar

Abstract

Background. Calibration (estimation of model parameters) compares model outcomes with observed outcomes and explores possible model parameter values to identify the set of values that provides the best fit to the data. The goodness-of-fit (GOF) criterion quantifies the difference between model and observed outcomes. There is no consensus on the most appropriate GOF criterion, because a direct performance comparison of GOF criteria in model calibration is lacking. Methods. We systematically compared the performance of commonly used GOF criteria (sum of squared errors [SSE], Pearson chi-square, and a likelihood-based approach [Poisson and/or binomial deviance functions]) in the calibration of selected parameters of the MISCAN-Colon microsimulation model for colorectal cancer. The performance of each GOF criterion was assessed by comparing the 1) root mean squared prediction error (RMSPE) of the selected parameters, 2) computation time of the calibration procedure of various calibration scenarios, and 3) impact on estimated cost-effectiveness ratios. Results. The likelihood-based deviance resulted in the lowest RMSPE in 4 of 6 calibration scenarios and was close to best in the other 2. The SSE had a 25 times higher RMSPE in a scenario with considerable differences in the values of observed outcomes, whereas the Pearson chi-square had a 60 times higher RMSPE in a scenario with multiple studies measuring the same outcome. In all scenarios, the SSE required the most computation time. The likelihood-based approach estimated the cost-effectiveness ratio most accurately (up to −0.15% relative difference versus 0.44% [SSE] and 13% [Pearson chi-square]). Conclusions. The likelihood-based deviance criteria lead to accurate estimation of parameters under various circumstances. These criteria are recommended for calibration in microsimulation disease models in contrast with other commonly used criteria.

Suggested Citation

  • Alex van der Steen & Joost van Rosmalen & Sonja Kroep & Frank van Hees & Ewout W. Steyerberg & Harry J. de Koning & Marjolein van Ballegooijen & Iris Lansdorp-Vogelaar, 2016. "Calibrating Parameters for Microsimulation Disease Models," Medical Decision Making, , vol. 36(5), pages 652-665, July.
    • Handle: RePEc:sae:medema:v:36:y:2016:i:5:p:652-665
    DOI: 10.1177/0272989X16636851
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0272989X16636851
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0272989X16636851?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Heij, Christiaan & de Boer, Paul & Franses, Philip Hans & Kloek, Teun & van Dijk, Herman K., 2004. "Econometric Methods with Applications in Business and Economics," OUP Catalogue, Oxford University Press, number 9780199268016.
    2. Alan Brennan & Stephen E. Chick & Ruth Davies, 2006. "A taxonomy of model structures for economic evaluation of health technologies," Health Economics, John Wiley & Sons, Ltd., vol. 15(12), pages 1295-1310, December.
    3. Neddermeijer, H.G. & van Oortmarssen, G.J. & Piersma, N. & Dekker, R. & Habbema, J.D.F., 2000. "Adaptive extensions of the Nelder and Mead Simplex Method for optimization of stochastic simulation models," Econometric Institute Research Papers EI 2000-22/A, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    4. Rutter, Carolyn M. & Miglioretti, Diana L. & Savarino, James E., 2009. "Bayesian Calibration of Microsimulation Models," Journal of the American Statistical Association, American Statistical Association, vol. 104(488), pages 1338-1350.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jing Voon Chen & Julia L. Higle & Michael Hintlian, 2018. "A systematic approach for examining the impact of calibration uncertainty in disease modeling," Computational Management Science, Springer, vol. 15(3), pages 541-561, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Franses, Ph.H.B.F. & van Oest, R.D., 2006. "Testing changes in consumer confidence indicators," Econometric Institute Research Papers EI 2006-18, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    2. Michael S. Willis & Andreas Nilsson & Cheryl A. Neslusan, 2025. "A Review of Heterogeneity in Comparative Economic Analysis, with Specific Considerations for the Decentralized US Setting and Patient-Centered Care," PharmacoEconomics, Springer, vol. 43(6), pages 601-616, June.
    3. Blum, Bianca, 2018. "Ausgestaltung einer Steuerpolitik zur Förderung von LED-Beleuchtung," The Constitutional Economics Network Working Papers 01-2018, University of Freiburg, Department of Economic Policy and Constitutional Economic Theory.
    4. Bjarne Jensen & Paul Boer & Jan Daal & Peter Jensen, 2011. "Global restrictions on the parameters of the CDES indirect utility function," Journal of Economics, Springer, vol. 102(3), pages 217-235, April.
    5. Hassan Belkacem Ghassan & Abdelkrim Ahmed Guendouz, 2019. "Panel modeling of z-score: evidence from Islamic and conventional Saudi banks," International Journal of Islamic and Middle Eastern Finance and Management, Emerald Group Publishing Limited, vol. 12(3), pages 448-468, July.
    6. Odolinski, Kristofer & Boysen, Hans E., 2018. "Railway line capacity utilisation and its impact on maintenance costs," Working papers in Transport Economics 2018:10, CTS - Centre for Transport Studies Stockholm (KTH and VTI), revised 30 Oct 2018.
    7. Heß, Michael (Ed.) & Schlieter, Hannes (Ed.), 2014. "Modellierung im Gesundheitswesen: Tagungsband des Workshops im Rahmen der Modellierung 2014," ICB Research Reports 57, University Duisburg-Essen, Institute for Computer Science and Business Information Systems (ICB).
    8. Rezitis Anthony N. & Rokopanos Andreas, 2019. "Asymmetric Price Transmission along the European Food Supply Chain and the CAP Health Check: a Panel Vector Error Correction Approach," Journal of Agricultural & Food Industrial Organization, De Gruyter, vol. 17(2), pages 1-20, November.
    9. Matthew Franklin & Sebastian Hinde & Rachael Maree Hunter & Gerry Richardson & William Whittaker, 2024. "Is Economic Evaluation and Care Commissioning Focused on Achieving the Same Outcomes? Resource-Allocation Considerations and Challenges Using England as a Case Study," Applied Health Economics and Health Policy, Springer, vol. 22(4), pages 435-445, July.
    10. Marion Rauner & Michaela Schaffhauser-Linzatti & Helmut Niessner, 2012. "Resource planning for ambulance services in mass casualty incidents: a DES-based policy model," Health Care Management Science, Springer, vol. 15(3), pages 254-269, September.
    11. Ardia, David & Hoogerheide, Lennart F. & van Dijk, Herman K., 2009. "Adaptive Mixture of Student-t Distributions as a Flexible Candidate Distribution for Efficient Simulation: The R Package AdMit," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 29(i03).
    12. Ivan Jericevich & Murray McKechnie & Tim Gebbie, 2021. "Calibrating an adaptive Farmer-Joshi agent-based model for financial markets," Papers 2104.09863, arXiv.org.
    13. Daisuke Goto & Ya-Chen Tina Shih & Pascal Lecomte & Melvin Olson & Chukwukadibia Udeze & Yujin Park & C. Daniel Mullins, 2017. "Regression-Based Approaches to Patient-Centered Cost-Effectiveness Analysis," PharmacoEconomics, Springer, vol. 35(7), pages 685-695, July.
    14. John Graves & Shawn Garbett & Zilu Zhou & Jonathan S. Schildcrout & Josh Peterson, 2021. "Comparison of Decision Modeling Approaches for Health Technology and Policy Evaluation," Medical Decision Making, , vol. 41(4), pages 453-464, May.
    15. Amaral, Hudson & Iquiapaza, Robert & Tomaz, Wesley & Bertucci, Luiz, 2008. "Governança corporativa e divulgação de relatórios financeiros anuais [Corporate governance and release of annual financial statements]," MPRA Paper 9068, University Library of Munich, Germany.
    16. Vaz, Thalita A. & Machado, Sérgio J. & Bortoluzzo, Adriana B., 2011. "Estimation of Conversion Rates into Annuities: A Brazilian Perspective," Insper Working Papers wpe_249, Insper Working Paper, Insper Instituto de Ensino e Pesquisa.
    17. Ortiz-Barrios, Miguel & Arias-Fonseca, Sebastián & Ishizaka, Alessio & Barbati, Maria & Avendaño-Collante, Betty & Navarro-Jiménez, Eduardo, 2023. "Artificial intelligence and discrete-event simulation for capacity management of intensive care units during the Covid-19 pandemic: A case study," Journal of Business Research, Elsevier, vol. 160(C).
    18. Thalita A. Vaz & Sérgio J. Machado & Adriana B. Bortoluzzo, 2011. "Estimation of Conversion Rates into Annuities: A Brazilian Perspective," Business and Economics Working Papers 139, Unidade de Negocios e Economia, Insper.
    19. Ka Keat Lim & Rositsa Koleva-Kolarova & Julia Fox-Rushby, 2022. "A Comparison of the Content and Consistency of Methodological Quality and Transferability Checklists for Reviewing Model-Based Economic Evaluations," PharmacoEconomics, Springer, vol. 40(10), pages 989-1003, October.
    20. Kevin Marsh & Ceri Phillips & Richard Fordham & Evelina Bertranou & Janine Hale, 2012. "Estimating cost-effectiveness in public health: a summary of modelling and valuation methods," Health Economics Review, Springer, vol. 2(1), pages 1-6, December.

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:medema:v:36:y:2016:i:5:p:652-665. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.