Natasha K. Stout (Department of Ambulatory Care and Prevention, Harvard Medical School/Harvard Pilgrim Health Care, Boston, Massachusetts, USA Program in Health Decision Science, Department of Health Policy and Management, Harvard School of Public Health, Boston, Massachusetts, USA) Amy B. Knudsen (Institute for Technology Assessment, Massachusetts General Hospital, Boston, Massachusetts, USA Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts, USA) Chung Yin. Kong (Institute for Technology Assessment, Massachusetts General Hospital, Boston, Massachusetts, USA Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts, USA) Pamela M. McMahon (Institute for Technology Assessment, Massachusetts General Hospital, Boston, Massachusetts, USA Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts, USA) G. Scott. Gazelle (Program in Health Decision Science, Department of Health Policy and Management, Harvard School of Public Health, Boston, Massachusetts, USA Institute for Technology Assessment, Massachusetts General Hospital, Boston, Massachusetts, USA Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts, USA)
Abstract
Increasingly, computer simulation models are used for economic and policy evaluation in cancer prevention and control. A model's predictions of key outcomes, such as screening effectiveness, depend on the values of unobservable natural history parameters. Calibration is the process of determining the values of unobservable parameters by constraining model output to replicate observed data. Because there are many approaches for model calibration and little consensus on best practices, we surveyed the literature to catalogue the use and reporting of these methods in cancer simulation models. We conducted a MEDLINE search (1980 through 2006) for articles on cancer-screening models and supplemented search results with articles from our personal reference databases. For each article, two authors independently abstracted pre-determined items using a standard form. Data items included cancer site, model type, methods used for determination of unobservable parameter values and description of any calibration protocol. All authors reached consensus on items of disagreement. Reviews and non-cancer models were excluded. Articles describing analytical models, which estimate parameters with statistical approaches (e.g. maximum likelihood) were catalogued separately. Models that included unobservable parameters were analysed and classified by whether calibration methods were reported and if so, the methods used. The review process yielded 154 articles that met our inclusion criteria and, of these, we concluded that 131 may have used calibration methods to determine model parameters. Although the term 'calibration' was not always used, descriptions of calibration or 'model fitting' were found in 50% (n - 66) of the articles, with an additional 16% (n - 21) providing a reference to methods. Calibration target data were identified in nearly all of these articles. Other methodological details, such as the goodness-of-fit metric, were discussed in 54% (n - 47 of 87) of the articles reporting calibration methods, while few details were provided on the algorithms used to search the parameter space. Our review shows that the use of cancer simulation modelling is increasing, although thorough descriptions of calibration procedures are rare in the published literature for these models. Calibration is a key component of model development and is central to the validity and credibility of subsequent analyses and inferences drawn from model predictions. To aid peer-review and facilitate discussion of modelling methods, we propose a standardized Calibration Reporting Checklist for model documentation.
Download Info
To download:
If you experience problems downloading a file, check if you have the
proper application to
view it first. Information about this may be contained
in the File-Format links below. In case of further problems read
the IDEAS help
page. Note that these files are not on the IDEAS
site. Please be patient as the files may be large.
As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.
Publisher Info
Article provided by Wolters Kluwer Health | Adis in its journal PharmacoEconomics.
For technical questions regarding this item, or to correct its listing, contact: (Remco Bouckaert).
Related research
Keywords:
Find related papers by JEL classification: C - Mathematical and Quantitative Methods D - Microeconomics I - Health, Education, and Welfare Z - Other Special Topics I1 - Health, Education, and Welfare - - Health I19 - Health, Education, and Welfare - - Health - - - Other I18 - Health, Education, and Welfare - - Health - - - Government Policy; Regulation; Public Health I11 - Health, Education, and Welfare - - Health - - - Analysis of Health Care Markets