IDEAS home Printed from https://ideas.repec.org/a/sae/medema/v33y2013i7p880-890.html
   My bibliography  Save this article

Linear Regression Metamodeling as a Tool to Summarize and Present Simulation Model Results

Author

Listed:
  • Hawre Jalal
  • Bryan Dowd
  • François Sainfort
  • Karen M. Kuntz

Abstract

Background/Objective. Modelers lack a tool to systematically and clearly present complex model results, including those from sensitivity analyses. The objective was to propose linear regression metamodeling as a tool to increase transparency of decision analytic models and better communicate their results. Methods. We used a simplified cancer cure model to demonstrate our approach. The model computed the lifetime cost and benefit of 3 treatment options for cancer patients. We simulated 10,000 cohorts in a probabilistic sensitivity analysis (PSA) and regressed the model outcomes on the standardized input parameter values in a set of regression analyses. We used the regression coefficients to describe measures of sensitivity analyses, including threshold and parameter sensitivity analyses. We also compared the results of the PSA to deterministic full-factorial and one-factor-at-a-time designs. Results. The regression intercept represented the estimated base-case outcome, and the other coefficients described the relative parameter uncertainty in the model. We defined simple relationships that compute the average and incremental net benefit of each intervention. Metamodeling produced outputs similar to traditional deterministic 1-way or 2-way sensitivity analyses but was more reliable since it used all parameter values. Conclusions. Linear regression metamodeling is a simple, yet powerful, tool that can assist modelers in communicating model characteristics and sensitivity analyses.

Suggested Citation

  • Hawre Jalal & Bryan Dowd & François Sainfort & Karen M. Kuntz, 2013. "Linear Regression Metamodeling as a Tool to Summarize and Present Simulation Model Results," Medical Decision Making, , vol. 33(7), pages 880-890, October.
  • Handle: RePEc:sae:medema:v:33:y:2013:i:7:p:880-890
    DOI: 10.1177/0272989X13492014
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0272989X13492014
    Download Restriction: no

    File URL: https://libkey.io/10.1177/0272989X13492014?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. M. D. Stevenson & J. Oakley & J. B. Chilcott, 2004. "Gaussian Process Modeling in Conjunction with Individual Patient Simulation Modeling: A Case Study Describing the Calculation of Cost-Effectiveness Ratios for the Treatment of Established Osteoporosis," Medical Decision Making, , vol. 24(1), pages 89-100, January.
    2. Kleijnen, Jack P. C., 2005. "An overview of the design and analysis of simulation experiments for sensitivity analysis," European Journal of Operational Research, Elsevier, vol. 164(2), pages 287-300, July.
    3. Kleijnen, Jack P. C. & Sargent, Robert G., 2000. "A methodology for fitting and validating metamodels in simulation," European Journal of Operational Research, Elsevier, vol. 120(1), pages 14-29, January.
    4. Doug Coyle & Jeremy Oakley, 2008. "Estimating the expected value of partial perfect information: a review of methods," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 9(3), pages 251-259, August.
    5. Jack P.C. Kleijnen, 2015. "Design and Analysis of Simulation Experiments," International Series in Operations Research and Management Science, Springer, edition 2, number 978-3-319-18087-8, September.
    6. Robert W. Blanning, 1974. "The Sources and Uses of Sensitivity Information," Interfaces, INFORMS, vol. 4(4), pages 32-38, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Radaideh, Majdi I. & Kozlowski, Tomasz, 2020. "Surrogate modeling of advanced computer simulations using deep Gaussian processes," Reliability Engineering and System Safety, Elsevier, vol. 195(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Poropudas, Jirka & Virtanen, Kai, 2011. "Simulation metamodeling with dynamic Bayesian networks," European Journal of Operational Research, Elsevier, vol. 214(3), pages 644-655, November.
    2. Shi, Wen & Liu, Zhixue & Shang, Jennifer & Cui, Yujia, 2013. "Multi-criteria robust design of a JIT-based cross-docking distribution center for an auto parts supply chain," European Journal of Operational Research, Elsevier, vol. 229(3), pages 695-706.
    3. Jack P. C. Kleijnen & Susan M. Sanchez & Thomas W. Lucas & Thomas M. Cioppa, 2005. "State-of-the-Art Review: A User’s Guide to the Brave New World of Designing Simulation Experiments," INFORMS Journal on Computing, INFORMS, vol. 17(3), pages 263-289, August.
    4. Kleijnen, Jack P.C., 2009. "Kriging metamodeling in simulation: A review," European Journal of Operational Research, Elsevier, vol. 192(3), pages 707-716, February.
    5. Kleijnen, Jack P.C., 2017. "Regression and Kriging metamodels with their experimental designs in simulation: A review," European Journal of Operational Research, Elsevier, vol. 256(1), pages 1-16.
    6. Strang, Kenneth David, 2012. "Importance of verifying queue model assumptions before planning with simulation software," European Journal of Operational Research, Elsevier, vol. 218(2), pages 493-504.
    7. Jin, Ding & Hedtrich, Johannes & Henning, Christian, 2018. "Applying Meta modeling for extended CGE-modeling: Sample techniques and potential application," Conference papers 332947, Purdue University, Center for Global Trade Analysis, Global Trade Analysis Project.
    8. Scott L. Rosen & Christopher P. Saunders & Samar K Guharay, 2015. "A Structured Approach for Rapidly Mapping Multilevel System Measures via Simulation Metamodeling," Systems Engineering, John Wiley & Sons, vol. 18(1), pages 87-101, January.
    9. Shi, Wen & Shang, Jennifer & Liu, Zhixue & Zuo, Xiaolu, 2014. "Optimal design of the auto parts supply chain for JIT operations: Sequential bifurcation factor screening and multi-response surface methodology," European Journal of Operational Research, Elsevier, vol. 236(2), pages 664-676.
    10. Marrel, Amandine & Iooss, Bertrand & Van Dorpe, François & Volkova, Elena, 2008. "An efficient methodology for modeling complex computer codes with Gaussian processes," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4731-4744, June.
    11. Happe, Kathrin & Kellermann, Konrad & Balmann, Alfons, 2006. "Agent-based analysis of agricultural policies: An illustration of the agricultural policy simulator AgriPoliS, its adaptation and behavior," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 11(1).
    12. Kleijnen, Jack P.C. & Mehdad, E., 2012. "Kriging in Multi-response Simulation, including a Monte Carlo Laboratory (Replaced by 2014-012)," Other publications TiSEM cf311469-5f8c-4c1e-ad4f-6, Tilburg University, School of Economics and Management.
    13. Reis dos Santos, M. Isabel & Porta Nova, Acacio M.O., 2006. "Statistical fitting and validation of non-linear simulation metamodels: A case study," European Journal of Operational Research, Elsevier, vol. 171(1), pages 53-63, May.
    14. Plischke, Elmar & Borgonovo, Emanuele & Smith, Curtis L., 2013. "Global sensitivity measures from given data," European Journal of Operational Research, Elsevier, vol. 226(3), pages 536-550.
    15. Hachicha, Wafik & Ammeri, Ahmed & Masmoudi, Faouzi & Chachoub, Habib, 2010. "A comprehensive literature classification of simulation optimisation methods," MPRA Paper 27652, University Library of Munich, Germany.
    16. Jin, Ding & Hedtrich, Johannes & Henning, Christian H. C. A., 2018. "Applying meta-modeling for extended CGE-modeling: Sampling techniques and potential application," Working Papers of Agricultural Policy WP2018-03, University of Kiel, Department of Agricultural Economics, Chair of Agricultural Policy.
    17. Michael C. Fu & Huashuai Qu, 2014. "Regression Models Augmented with Direct Stochastic Gradient Estimators," INFORMS Journal on Computing, INFORMS, vol. 26(3), pages 484-499, August.
    18. Iooss, Bertrand & Van Dorpe, François & Devictor, Nicolas, 2006. "Response surfaces and sensitivity analyses for an environmental model of dose calculations," Reliability Engineering and System Safety, Elsevier, vol. 91(10), pages 1241-1251.
    19. Imry Rosenbaum & Jeremy Staum, 2017. "Multilevel Monte Carlo Metamodeling," Operations Research, INFORMS, vol. 65(4), pages 1062-1077, August.
    20. Rennen, G. & Husslage, B.G.M. & van Dam, E.R. & den Hertog, D., 2009. "Nested Maximin Latin Hypercube Designs," Discussion Paper 2009-06, Tilburg University, Center for Economic Research.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:medema:v:33:y:2013:i:7:p:880-890. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.