IDEAS home Printed from
   My bibliography  Save this paper

Moving Beyond Comparative Validation: Predictive Abilities of APPSIM's Health Module


  • Sharyn Lymer

    (NATSEM, University of Canberra)

  • Alan Duncan

    (NATSEM, University of Canberra)

  • Laurie Brown

    (NATSEM, University of Canberra)


In the development of a dynamic microsimulation model, validation lends credibility to the model, making it more likely to be accepted by policy makers. In the validation of multi-module models such as the Australian Population and Policy Simulation model (APPSIM) currently under development at the National Centre for Social and Economic Modelling (NATSEM), one is faced with additional validation challenges. Not only do the specific characteristics of the module under consideration need validating but so too do the data inputs from earlier developed modules. Moreover, validation needs to account for dynamic interactions with the other modules in the model system. This paper considers the validation of the health module of APPSIM, which is itself highly reliant on prerequisite modules such as education and earnings. Commonly used comparative validation methods are used. In addition, as part of the validation of the health module, issues related to variable inputs are disentangled from the health prediction models. To achieve this, a version of the APPSIM health module is developed in SAS and run against the Australian longitudinal HILDA dataset between 2006 and 2008. This provides an alternate data source for the socio-economic variables used in the model, providing health outcomes to assess the equation?s predictive quality. Since these data are not used in the development of the prediction equations, the proposed method provides an independent dataset for testing the module?s generalisability. Using this method, it is possible to verify both the baseline imputation and the transition equations used in the dynamic simulation model. The validation of the equations? predictive qualities includes „confusion tables? to consider at a simple level the predictive accuracy of the model, precision measures looking at the ratio of true positive results to all positive results, and ROC curves to consider the true positives against the false positives. As an illustration, a validation of predictions of being overweight or obese from the APPSIM health module is presented.

Suggested Citation

  • Sharyn Lymer & Alan Duncan & Laurie Brown, 2011. "Moving Beyond Comparative Validation: Predictive Abilities of APPSIM's Health Module," NATSEM Working Paper Series 11/10, University of Canberra, National Centre for Social and Economic Modelling.
  • Handle: RePEc:cba:wpaper:wp1110

    Download full text from publisher

    File URL:
    Download Restriction: no

    References listed on IDEAS

    1. Cathal O'Donoghue, 2001. "Dynamic Microsimulation: A Methodological Survey," Brazilian Electronic Journal of Economics, Department of Economics, Universidade Federal de Pernambuco, vol. 4(2), December.
    2. Pudney, Stephen & Sutherland, Holly, 1994. "How reliable are microsimulation results? : An analysis of the role of sampling error in a U.K. tax-benefit model," Journal of Public Economics, Elsevier, vol. 53(3), pages 327-365, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Stephen Pudney & Ruth Hancock & Holly Sutherland, 2006. "Simulating the Reform of Means‐tested Benefits with Endogenous Take‐up and Claim Costs," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 68(2), pages 135-166, April.
    2. Tim Goedemé & Karel Van den Bosch & Lina Salanauskaite & Gerlinde Verbist, 2013. "Testing the Statistical Significance of Microsimulation Results: Often Easier than You Think. A Technical Note," ImPRovE Working Papers 13/10, Herman Deleeck Centre for Social Policy, University of Antwerp.
    3. Nelissen, J.H.M., 1994. "Gedragseffecten en kringloopeffecten in microsimulatiemodellen," WORC Paper 94.11.060/2, Tilburg University, Work and Organization Research Centre.
    4. Mercader-Prats, Magda, 1997. "On the distributive and incentive effects of the Spanish income tax: A comparison of 1980 and 1994," European Economic Review, Elsevier, vol. 41(3-5), pages 609-617, April.
    5. Karine Briard, 2009. "Un modèle de carrières types dynamiques pondérées pour le régime général d’assurance vieillesse : une application aux conséquences de la réforme de 2003," Économie et Prévision, Programme National Persée, vol. 187(1), pages 47-64.
    6. David Piachaud & Holly Sutherland, 2000. "How Effective is the British Governments Attempt to Reduce Child Poverty?," CASE Papers case38, Centre for Analysis of Social Exclusion, LSE.
    7. van Sonsbeek, J.M. & Gradus, R.H.J.M., 2006. "A microsimulation analysis of the 2006 regime change in the Dutch disability scheme," Economic Modelling, Elsevier, vol. 23(3), pages 427-456, May.
    8. Herwig Immervoll, 2006. "Fiscal Drag – An Automatic Stabiliser?," Research in Labor Economics, in: Micro-Simulation in Action, pages 141-163, Emerald Group Publishing Limited.
    9. John Creedy & Guyonne Kalb & Hsein Kew, 2007. "Confidence Intervals For Policy Reforms In Behavioural Tax Microsimulation Modelling," Bulletin of Economic Research, Wiley Blackwell, vol. 59(1), pages 37-65, January.
    10. Heiko Müller & Caren Sureth, 2009. "Income tax statistics analysis: A comparison of microsimulation versus group simulation," International Journal of Microsimulation, International Microsimulation Association, vol. 2(1), pages 32-48.
    11. Roberto Leombruni & Matteo Richiardi, 2006. "LABORsim: An Agent-Based Microsimulation of Labour Supply – An Application to Italy," Computational Economics, Springer;Society for Computational Economics, vol. 27(1), pages 63-88, February.
    12. Francesco Figari & Maria Iacovou & Alexandra Skew & Holly Sutherland, 2012. "Approximations to the Truth: Comparing Survey and Microsimulation Approaches to Measuring Income for Social Indicators," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 105(3), pages 387-407, February.
    13. Vaqar Ahmed & Cathal O'Donoghue, 2009. "Redistributive Effect of Personal Income Taxation in Pakistan," Working Papers 0143, National University of Ireland Galway, Department of Economics, revised 2009.
    14. François Bourguignon & Amedeo Spadaro, 2006. "Microsimulation as a tool for evaluating redistribution policies," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 4(1), pages 77-106, April.
    15. Jinjing Li & Cathal O'Donoghue, 2013. "A survey of dynamic microsimulation models: uses, model structure and methodology," International Journal of Microsimulation, International Microsimulation Association, vol. 6(2), pages 3-55.
    16. John Creedy & Guyonne Kalb, 2005. "Behavioural Microsimulation Modelling With the Melbourne Institute Tax and Transfer Simulator(MITTS) : Uses and Extensions," Department of Economics - Working Papers Series 932, The University of Melbourne.
    17. Callan T & O'donoghue C & Sutherland H, 1999. "Comparative Analysis of Basic Income Proposals: UK and Ireland," Microsimulation Unit Research Notes MU/RN/31, Microsimulation Unit at the Institute for Social and Economic Research.
    18. Newbery, David M., 1997. "Optimal tax rates and tax design during systemic reform," Journal of Public Economics, Elsevier, vol. 63(2), pages 177-206, January.
    19. Immervoll, Herwig, 2002. "The distribution of average and marginal effective tax rates in European Union Member States," EUROMOD Working Papers EM2/02, EUROMOD at the Institute for Social and Economic Research.
    20. O'Donoghue, Cathal & Immervoll, Herwig, 2001. "Towards a multi purpose framework for tax benefit microsimulation," EUROMOD Working Papers EM2/01, EUROMOD at the Institute for Social and Economic Research.

    More about this item


    Health; APPSIM;


    Access and download statistics


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cba:wpaper:wp1110. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Peter Trueman (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.