IDEAS home Printed from
MyIDEAS: Login to save this article or follow this journal

Calibrating Models in Economic Evaluation: A Comparison of Alternative Measures of Goodness of Fit, Parameter Search Strategies and Convergence Criteria

  • Jonathan Karnon

    (University of Adelaide, Adelaide, South Australia, Australia)

  • Tazio Vanni

    (London School of Hygiene and Tropical Medicine, London, UK)

Registered author(s):

    Background: The importance of assessing the accuracy of health economic decision models is widely recognized. Many applied decision models (implicitly) assume that the process of identifying relevant values for a model's input parameters is sufficient to prove the model's accuracy. The selection of infeasible combinations of input parameter values is most likely in the context of probabilistic sensitivity analysis (PSA), where parameter values are drawn from independently specified probability distributions for each model parameter. Model calibration involves the identification of input parameter values that produce model output parameters that best predict observed data. Methods: An empirical comparison of three key calibration issues is presented: the applied measure of goodness of fit (GOF); the search strategy for selecting sets of input parameter values; and the convergence criteria for determining acceptable GOF. The comparisons are presented in the context of probabilistic calibration, a widely applicable approach to calibration that can be easily integrated with PSA. The appendix provides a user's guide to probabilistic calibration, with the reader invited to download the Microsoft Excel-based model reported in this article. Results: The calibrated models consistently provided higher mean estimates of the models' output parameter, illustrating the potential gain in accuracy derived from calibrating decision models. Model uncertainty was also reduced. The chi-squared GOF measure differentiated between the accuracy of different parameter sets to a far greater degree than the likelihood GOF measure. The guided search strategy produced higher mean estimates of the models' output parameter, as well as a narrower range of predicted output values, which may reflect greater precision in the identification of candidate parameter sets or more limited coverage of the parameter space. The broader convergence threshold resulted in lower mean estimates of the models' output, and slightly wider ranges, which were closer to the outputs associated with the non-calibrated approach. Conclusions: Probabilistic calibration provides a broadly applicable method that will improve the relevance of health economic decision models, and simultaneously reduce model uncertainty. The analyses reported in this paper inform the more efficient and accurate application of calibration methods for health economic decision models.

    If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

    File URL:
    Download Restriction: Pay per view

    File URL:
    Download Restriction: Pay per view

    As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.

    Article provided by Springer Healthcare | Adis in its journal PharmacoEconomics.

    Volume (Year): 29 (2011)
    Issue (Month): 1 ()
    Pages: 51-62

    in new window

    Handle: RePEc:wkh:phecon:v:29:y:2011:i:1:p:51-62
    Contact details of provider: Web page:

    No references listed on IDEAS
    You can help add them by filling out this form.

    This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

    When requesting a correction, please mention this item's handle: RePEc:wkh:phecon:v:29:y:2011:i:1:p:51-62. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Dave Dustin)

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If references are entirely missing, you can add them using this form.

    If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.