IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v226y2013i3p536-550.html
   My bibliography  Save this article

Global sensitivity measures from given data

Author

Listed:
  • Plischke, Elmar
  • Borgonovo, Emanuele
  • Smith, Curtis L.

Abstract

Simulation models support managers in the solution of complex problems. International agencies recommend uncertainty and global sensitivity methods as best practice in the audit, validation and application of scientific codes. However, numerical complexity, especially in the presence of a high number of factors, induces analysts to employ less informative but numerically cheaper methods. This work introduces a design for estimating global sensitivity indices from given data (including simulation input–output data), at the minimum computational cost. We address the problem starting with a statistic based on the L1-norm. A formal definition of the estimators is provided and corresponding consistency theorems are proved. The determination of confidence intervals through a bias-reducing bootstrap estimator is investigated. The strategy is applied in the identification of the key drivers of uncertainty for the complex computer code developed at the National Aeronautics and Space Administration (NASA) assessing the risk of lunar space missions. We also introduce a symmetry result that enables the estimation of global sensitivity measures to datasets produced outside a conventional input–output functional framework.

Suggested Citation

  • Plischke, Elmar & Borgonovo, Emanuele & Smith, Curtis L., 2013. "Global sensitivity measures from given data," European Journal of Operational Research, Elsevier, vol. 226(3), pages 536-550.
  • Handle: RePEc:eee:ejores:v:226:y:2013:i:3:p:536-550
    DOI: 10.1016/j.ejor.2012.11.047
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221712008995
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ejor.2012.11.047?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Kleijnen, Jack P.C. & Deflandre, David, 2006. "Validation of regression metamodels in simulation: Bootstrap approach," European Journal of Operational Research, Elsevier, vol. 170(1), pages 120-131, April.
    2. Kleijnen, Jack P. C., 2005. "An overview of the design and analysis of simulation experiments for sensitivity analysis," European Journal of Operational Research, Elsevier, vol. 164(2), pages 287-300, July.
    3. Borgonovo, E., 2007. "A new uncertainty importance measure," Reliability Engineering and System Safety, Elsevier, vol. 92(6), pages 771-784.
    4. Manel Baucells & Rakesh Sarin, 2007. "Evaluating Time Streams of Income: Discounting What?," Theory and Decision, Springer, vol. 63(2), pages 95-120, September.
    5. Gordon B. Hazen & Min Huang, 2006. "Parametric Sensitivity Analysis Using Large-Sample Approximate Bayesian Posterior Distributions," Decision Analysis, INFORMS, vol. 3(4), pages 208-219, December.
    6. Andrea Saltelli & Beatrice D’Hombres & Jochen Jesinghaus & Anna Manca & Massimiliano Mascherini & Michela Nardo & Michaela Saisana, 2011. "Indicators for European Union Policies. Business as Usual?," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 102(2), pages 197-207, June.
    7. Kleijnen, Jack P.C., 2009. "Kriging metamodeling in simulation: A review," European Journal of Operational Research, Elsevier, vol. 192(3), pages 707-716, February.
    8. van Beers, Wim C.M. & Kleijnen, Jack P.C., 2008. "Customized sequential designs for random simulation experiments: Kriging metamodeling and bootstrapping," European Journal of Operational Research, Elsevier, vol. 186(3), pages 1099-1113, May.
    9. Oakley, Jeremy E. & Brennan, Alan & Tappenden, Paul & Chilcott, Jim, 2010. "Simulation sample sizes for Monte Carlo partial EVPI calculations," Journal of Health Economics, Elsevier, vol. 29(3), pages 468-477, May.
    10. Ratto, M. & Pagano, A. & Young, P.C., 2009. "Non-parametric estimation of conditional moments for sensitivity analysis," Reliability Engineering and System Safety, Elsevier, vol. 94(2), pages 237-243.
    11. Harvey M. Wagner, 1995. "Global Sensitivity Analysis," Operations Research, INFORMS, vol. 43(6), pages 948-969, December.
    12. Plischke, Elmar, 2012. "An adaptive correlation ratio method using the cumulative sum of the reordered output," Reliability Engineering and System Safety, Elsevier, vol. 107(C), pages 149-156.
    13. Hong Wan & Bruce E. Ankenman & Barry L. Nelson, 2010. "Improving the Efficiency and Efficacy of Controlled Sequential Bifurcation for Simulation Factor Screening," INFORMS Journal on Computing, INFORMS, vol. 22(3), pages 482-492, August.
    14. Tarantola, S. & Gatelli, D. & Mara, T.A., 2006. "Random balance designs for the estimation of first order global sensitivity indices," Reliability Engineering and System Safety, Elsevier, vol. 91(6), pages 717-727.
    15. Kleijnen, J. P. C., 2001. "Ethical issues in modeling: Some reflections," European Journal of Operational Research, Elsevier, vol. 130(1), pages 223-230, April.
    16. Stephen E. Chick, 2001. "Input Distribution Selection for Simulation Experiments: Accounting for Input Uncertainty," Operations Research, INFORMS, vol. 49(5), pages 744-758, October.
    17. Sudret, Bruno, 2008. "Global sensitivity analysis using polynomial chaos expansions," Reliability Engineering and System Safety, Elsevier, vol. 93(7), pages 964-979.
    18. Kleijnen, J.P.C. & Bettonvil, B.W.M., 1997. "Searching for important factors in simulation models with many factors : Sequential bifurcation," Other publications TiSEM be826993-22f9-4cb3-89df-3, Tilburg University, School of Economics and Management.
    19. Saltelli A. & Tarantola S., 2002. "On the Relative Importance of Input Factors in Mathematical Models: Safety Assessment for Nuclear Waste Disposal," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 702-709, September.
    20. Kleijnen, J.P.C., 2001. "Ethical issues in modeling : Some reflections," Other publications TiSEM 9da35476-71ca-4696-94f7-0, Tilburg University, School of Economics and Management.
    21. Alison L. Gibbs & Francis Edward Su, 2002. "On Choosing and Bounding Probability Metrics," International Statistical Review, International Statistical Institute, vol. 70(3), pages 419-435, December.
    22. E. Borgonovo & C. L. Smith, 2011. "A Study of Interactions in the Risk Assessment of Complex Engineering Systems: An Application to Space PSA," Operations Research, INFORMS, vol. 59(6), pages 1461-1476, December.
    23. Liu, Qiao & Homma, Toshimitsu, 2009. "A new computational method of a moment-independent uncertainty importance measure," Reliability Engineering and System Safety, Elsevier, vol. 94(7), pages 1205-1211.
    24. Jeremy E. Oakley & Anthony O'Hagan, 2004. "Probabilistic sensitivity analysis of complex models: a Bayesian approach," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 66(3), pages 751-769, August.
    25. Jack P. C. Kleijnen, 2009. "Factor Screening in Simulation Experiments: Review of Sequential Bifurcation," International Series in Operations Research & Management Science, in: Christos Alexopoulos & David Goldsman & James R. Wilson (ed.), Advancing the Frontiers of Simulation, pages 153-167, Springer.
    26. Xu, C. & Gertner, G., 2007. "Extending a global sensitivity analysis technique to models with correlated parameters," Computational Statistics & Data Analysis, Elsevier, vol. 51(12), pages 5579-5590, August.
    27. Bettonvil, Bert & Kleijnen, Jack P. C., 1997. "Searching for important factors in simulation models with many factors: Sequential bifurcation," European Journal of Operational Research, Elsevier, vol. 96(1), pages 180-194, January.
    28. Jack P.C. Kleijnen, 2015. "Design and Analysis of Simulation Experiments," International Series in Operations Research and Management Science, Springer, edition 2, number 978-3-319-18087-8, December.
    29. Mark Strong & Jeremy E. Oakley & Jim Chilcott, 2012. "Managing structural uncertainty in health economic decision models: a discrepancy approach," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 61(1), pages 25-45, January.
    30. M. Saisana & A. Saltelli & S. Tarantola, 2005. "Uncertainty and sensitivity analysis techniques as tools for the quality assessment of composite indicators," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 168(2), pages 307-323, March.
    31. Xiaoqun Wang, 2006. "On the Effects of Dimension Reduction Techniques on Some High-Dimensional Problems in Finance," Operations Research, INFORMS, vol. 54(6), pages 1063-1078, December.
    32. Kenneth C. Lichtendahl & Samuel E. Bodily, 2010. "Preferences for Consumption Streams: Scale Invariance, Correlation Aversion, and Delay Aversion Under Mortality Risk," Operations Research, INFORMS, vol. 58(4-part-1), pages 985-997, August.
    33. Plischke, Elmar, 2010. "An effective algorithm for computing global sensitivity indices (EASI)," Reliability Engineering and System Safety, Elsevier, vol. 95(4), pages 354-360.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Borgonovo, Emanuele & Plischke, Elmar, 2016. "Sensitivity analysis: A review of recent advances," European Journal of Operational Research, Elsevier, vol. 248(3), pages 869-887.
    2. Mirko Ginocchi & Ferdinanda Ponci & Antonello Monti, 2021. "Sensitivity Analysis and Power Systems: Can We Bridge the Gap? A Review and a Guide to Getting Started," Energies, MDPI, vol. 14(24), pages 1-59, December.
    3. Emanuele Borgonovo & Gordon B. Hazen & Elmar Plischke, 2016. "A Common Rationale for Global Sensitivity Measures and Their Estimation," Risk Analysis, John Wiley & Sons, vol. 36(10), pages 1871-1895, October.
    4. Wei, Pengfei & Lu, Zhenzhou & Song, Jingwen, 2015. "Variable importance analysis: A comprehensive review," Reliability Engineering and System Safety, Elsevier, vol. 142(C), pages 399-432.
    5. Kleijnen, Jack P.C., 2017. "Regression and Kriging metamodels with their experimental designs in simulation: A review," European Journal of Operational Research, Elsevier, vol. 256(1), pages 1-16.
    6. Xuefei Lu & Alessandro Rudi & Emanuele Borgonovo & Lorenzo Rosasco, 2020. "Faster Kriging: Facing High-Dimensional Simulators," Operations Research, INFORMS, vol. 68(1), pages 233-249, January.
    7. Elmar Plischke & Emanuele Borgonovo, 2020. "Fighting the Curse of Sparsity: Probabilistic Sensitivity Measures From Cumulative Distribution Functions," Risk Analysis, John Wiley & Sons, vol. 40(12), pages 2639-2660, December.
    8. Manel Baucells & Emanuele Borgonovo, 2013. "Invariant Probabilistic Sensitivity Analysis," Management Science, INFORMS, vol. 59(11), pages 2536-2549, November.
    9. S. Cucurachi & E. Borgonovo & R. Heijungs, 2016. "A Protocol for the Global Sensitivity Analysis of Impact Assessment Models in Life Cycle Assessment," Risk Analysis, John Wiley & Sons, vol. 36(2), pages 357-377, February.
    10. Zhai, Qingqing & Yang, Jun & Zhao, Yu, 2014. "Space-partition method for the variance-based sensitivity analysis: Optimal partition scheme and comparative study," Reliability Engineering and System Safety, Elsevier, vol. 131(C), pages 66-82.
    11. Shi, Wen & Kleijnen, Jack P.C. & Liu, Zhixue, 2014. "Factor screening for simulation with multiple responses: Sequential bifurcation," European Journal of Operational Research, Elsevier, vol. 237(1), pages 136-147.
    12. Awad, Mahmoud, 2017. "Analyzing sensitivity measures using moment-matching technique," Reliability Engineering and System Safety, Elsevier, vol. 159(C), pages 90-99.
    13. Barry Anderson & Emanuele Borgonovo & Marzio Galeotti & Roberto Roson, 2014. "Uncertainty in Climate Change Modeling: Can Global Sensitivity Analysis Be of Help?," Risk Analysis, John Wiley & Sons, vol. 34(2), pages 271-293, February.
    14. Shi, Wen & Shang, Jennifer & Liu, Zhixue & Zuo, Xiaolu, 2014. "Optimal design of the auto parts supply chain for JIT operations: Sequential bifurcation factor screening and multi-response surface methodology," European Journal of Operational Research, Elsevier, vol. 236(2), pages 664-676.
    15. Allaire, Douglas L. & Willcox, Karen E., 2012. "A variance-based sensitivity index function for factor prioritization," Reliability Engineering and System Safety, Elsevier, vol. 107(C), pages 107-114.
    16. Plischke, Elmar, 2012. "An adaptive correlation ratio method using the cumulative sum of the reordered output," Reliability Engineering and System Safety, Elsevier, vol. 107(C), pages 149-156.
    17. Tianyang Wang & James S. Dyer & Warren J. Hahn, 2017. "Sensitivity analysis of decision making under dependent uncertainties using copulas," EURO Journal on Decision Processes, Springer;EURO - The Association of European Operational Research Societies, vol. 5(1), pages 117-139, November.
    18. Puppo, L. & Pedroni, N. & Maio, F. Di & Bersano, A. & Bertani, C. & Zio, E., 2021. "A Framework based on Finite Mixture Models and Adaptive Kriging for Characterizing Non-Smooth and Multimodal Failure Regions in a Nuclear Passive Safety System," Reliability Engineering and System Safety, Elsevier, vol. 216(C).
    19. Wei, Pengfei & Lu, Zhenzhou & Yuan, Xiukai, 2013. "Monte Carlo simulation for moment-independent sensitivity analysis," Reliability Engineering and System Safety, Elsevier, vol. 110(C), pages 60-67.
    20. Kleijnen, J.P.C., 2009. "Sensitivity Analysis of Simulation Models," Discussion Paper 2009-11, Tilburg University, Center for Economic Research.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:226:y:2013:i:3:p:536-550. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.