IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v248y2016i3p869-887.html
   My bibliography  Save this article

Sensitivity analysis: A review of recent advances

Author

Listed:
  • Borgonovo, Emanuele
  • Plischke, Elmar

Abstract

The solution of several operations research problems requires the creation of a quantitative model. Sensitivity analysis is a crucial step in the model building and result communication process. Through sensitivity analysis we gain essential insights on model behavior, on its structure and on its response to changes in the model inputs. Several interrogations are possible and several sensitivity analysis methods have been developed, giving rise to a vast and growing literature. We present an overview of available methods, structuring them into local and global methods. For local methods, we discuss Tornado diagrams, one way sensitivity functions, differentiation-based methods and scenario decomposition through finite change sensitivity indices, providing a unified view of the associated sensitivity measures. We then analyze global sensitivity methods, first discussing screening methods such as sequential bifurcation and the Morris method. We then address variance-based, moment-independent and value of information-based sensitivity methods. We discuss their formalization in a common rationale and present recent results that permit the estimation of global sensitivity measures by post-processing the sample generated by a traditional Monte Carlo simulation. We then investigate in detail the methodological issues concerning the crucial step of correctly interpreting the results of a sensitivity analysis. A classical example is worked out to illustrate some of the approaches.

Suggested Citation

  • Borgonovo, Emanuele & Plischke, Elmar, 2016. "Sensitivity analysis: A review of recent advances," European Journal of Operational Research, Elsevier, vol. 248(3), pages 869-887.
  • Handle: RePEc:eee:ejores:v:248:y:2016:i:3:p:869-887
    DOI: 10.1016/j.ejor.2015.06.032
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221715005469
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ejor.2015.06.032?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Lars Peter Hansen & Thomas J Sargent, 2014. "Beliefs, Doubts and Learning: Valuing Macroeconomic Risk," World Scientific Book Chapters, in: UNCERTAINTY WITHIN ECONOMIC MODELS, chapter 10, pages 331-377, World Scientific Publishing Co. Pte. Ltd..
    2. M. Avriel & A. C. Williams, 1970. "The Value of Information and Stochastic Programming," Operations Research, INFORMS, vol. 18(5), pages 947-954, October.
    3. Ford W. Harris, 1990. "How Many Parts to Make at Once," Operations Research, INFORMS, vol. 38(6), pages 947-950, December.
    4. C. C. Huang & I. Vertinsky & W. T. Ziemba, 1977. "Sharp Bounds on the Value of Perfect Information," Operations Research, INFORMS, vol. 25(1), pages 128-139, February.
    5. Tietje, Olaf, 2005. "Identification of a small reliable and efficient set of consistent scenarios," European Journal of Operational Research, Elsevier, vol. 162(2), pages 418-432, April.
    6. Zhang, Leigang & Lu, Zhenzhou & Cheng, Lei & Fan, Chongqing, 2014. "A new method for evaluating Borgonovo moment-independent importance measure with its application in an aircraft structure," Reliability Engineering and System Safety, Elsevier, vol. 132(C), pages 163-175.
    7. Dimitris Bertsimas & Melvyn Sim, 2004. "The Price of Robustness," Operations Research, INFORMS, vol. 52(1), pages 35-53, February.
    8. Borgonovo, E., 2007. "A new uncertainty importance measure," Reliability Engineering and System Safety, Elsevier, vol. 92(6), pages 771-784.
    9. Stein W. Wallace, 2000. "Decision Making Under Uncertainty: Is Sensitivity Analysis of Any Use?," Operations Research, INFORMS, vol. 48(1), pages 20-25, February.
    10. Storlie, Curtis B. & Swiler, Laura P. & Helton, Jon C. & Sallaberry, Cedric J., 2009. "Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models," Reliability Engineering and System Safety, Elsevier, vol. 94(11), pages 1735-1763.
    11. N. Ravi & Richard E. Wendell, 1989. "The Tolerance Approach to Sensitivity Analysis of Matrix Coefficients in Linear Programming," Management Science, INFORMS, vol. 35(9), pages 1106-1119, September.
    12. Borgonovo, E., 2010. "The reliability importance of components and prime implicants in coherent and non-coherent systems including total-order interactions," European Journal of Operational Research, Elsevier, vol. 204(3), pages 485-495, August.
    13. Richard E. Wendell, 1985. "The Tolerance Approach to Sensitivity Analysis in Linear Programming," Management Science, INFORMS, vol. 31(5), pages 564-578, May.
    14. Zhai, Qingqing & Yang, Jun & Zhao, Yu, 2014. "Space-partition method for the variance-based sensitivity analysis: Optimal partition scheme and comparative study," Reliability Engineering and System Safety, Elsevier, vol. 131(C), pages 66-82.
    15. E. Borgonovo & S. Tarantola & E. Plischke & M. D. Morris, 2014. "Transformations and invariance in the sensitivity analysis of computer experiments," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 76(5), pages 925-947, November.
    16. Kleijnen, Jack P.C., 2009. "Kriging metamodeling in simulation: A review," European Journal of Operational Research, Elsevier, vol. 192(3), pages 707-716, February.
    17. John D. C. Little, 1970. "Models and Managers: The Concept of a Decision Calculus," Management Science, INFORMS, vol. 16(8), pages 466-485, April.
    18. Zio, Enrico & Podofillini, Luca, 2006. "Accounting for components interactions in the differential importance measure," Reliability Engineering and System Safety, Elsevier, vol. 91(10), pages 1163-1174.
    19. Lars Peter Hansen, 2007. "Beliefs, Doubts and Learning: Valuing Economic Risk," NBER Working Papers 12948, National Bureau of Economic Research, Inc.
    20. Concha Bielza & Prakash P. Shenoy, 1999. "A Comparison of Graphical Techniques for Asymmetric Decision Problems," Management Science, INFORMS, vol. 45(11), pages 1552-1569, November.
    21. Zhai, Qingqing & Yang, Jun & Xie, Min & Zhao, Yu, 2014. "Generalized moment-independent importance measures based on Minkowski distance," European Journal of Operational Research, Elsevier, vol. 239(2), pages 449-455.
    22. Mark Strong & Jeremy E. Oakley, 2013. "An Efficient Method for Computing Single-Parameter Partial Expected Value of Perfect Information," Medical Decision Making, , vol. 33(6), pages 755-766, August.
    23. Beccacece, F. & Borgonovo, E., 2011. "Functional ANOVA, ultramodularity and monotonicity: Applications in multiattribute utility theory," European Journal of Operational Research, Elsevier, vol. 210(2), pages 326-335, April.
    24. Do Van, Phuc & Barros, Anne & Bérenguer, Christophe, 2008. "Reliability importance analysis of Markovian systems at steady state using perturbation analysis," Reliability Engineering and System Safety, Elsevier, vol. 93(11), pages 1605-1615.
    25. Harvey M. Wagner, 1995. "Global Sensitivity Analysis," Operations Research, INFORMS, vol. 43(6), pages 948-969, December.
    26. Plischke, Elmar, 2012. "An adaptive correlation ratio method using the cumulative sum of the reordered output," Reliability Engineering and System Safety, Elsevier, vol. 107(C), pages 149-156.
    27. Hong Wan & Bruce E. Ankenman & Barry L. Nelson, 2010. "Improving the Efficiency and Efficacy of Controlled Sequential Bifurcation for Simulation Factor Screening," INFORMS Journal on Computing, INFORMS, vol. 22(3), pages 482-492, August.
    28. Simon French, 2003. "Modelling, making inferences and making decisions: The roles of sensitivity analysis," TOP: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 11(2), pages 229-251, December.
    29. Tarantola, S. & Gatelli, D. & Mara, T.A., 2006. "Random balance designs for the estimation of first order global sensitivity indices," Reliability Engineering and System Safety, Elsevier, vol. 91(6), pages 717-727.
    30. Gregory C. Critchfield & Keith E. Willard, 1986. "Probabilistic Analysis of Decision Trees Using Monte Carlo Simulation," Medical Decision Making, , vol. 6(2), pages 85-92, June.
    31. Manel Baucells & Emanuele Borgonovo, 2013. "Invariant Probabilistic Sensitivity Analysis," Management Science, INFORMS, vol. 59(11), pages 2536-2549, November.
    32. James C. Felli & Gordon B. Hazen, 1998. "Sensitivity Analysis and the Expected Value of Perfect Information," Medical Decision Making, , vol. 18(1), pages 95-109, January.
    33. Xiaoqun Wang & Ian H. Sloan, 2011. "Quasi-Monte Carlo Methods in Financial Engineering: An Equivalence Principle and Dimension Reduction," Operations Research, INFORMS, vol. 59(1), pages 80-95, February.
    34. Bolado-Lavin, R. & Castaings, W. & Tarantola, S., 2009. "Contribution to the sample mean plot for graphical and numerical sensitivity analysis," Reliability Engineering and System Safety, Elsevier, vol. 94(6), pages 1041-1049.
    35. Van Groenendaal, Willem J. H. & Kleijnen, Jack P. C., 2002. "Deterministic versus stochastic sensitivity analysis in investment problems: An environmental case study," European Journal of Operational Research, Elsevier, vol. 141(1), pages 8-20, August.
    36. Sudret, Bruno, 2008. "Global sensitivity analysis using polynomial chaos expansions," Reliability Engineering and System Safety, Elsevier, vol. 93(7), pages 964-979.
    37. Borgonovo, E. & Peccati, L., 2004. "Sensitivity analysis in investment project evaluation," International Journal of Production Economics, Elsevier, vol. 90(1), pages 17-25, July.
    38. Ted G. Eschenbach, 1992. "Spiderplots versus Tornado Diagrams for Sensitivity Analysis," Interfaces, INFORMS, vol. 22(6), pages 40-46, December.
    39. Kleijnen, J.P.C. & Bettonvil, B.W.M., 1997. "Searching for important factors in simulation models with many factors : Sequential bifurcation," Other publications TiSEM be826993-22f9-4cb3-89df-3, Tilburg University, School of Economics and Management.
    40. Saltelli A. & Tarantola S., 2002. "On the Relative Importance of Input Factors in Mathematical Models: Safety Assessment for Nuclear Waste Disposal," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 702-709, September.
    41. Daniel Ellsberg, 1961. "Risk, Ambiguity, and the Savage Axioms," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 75(4), pages 643-669.
    42. E. Borgonovo & C. L. Smith, 2011. "A Study of Interactions in the Risk Assessment of Complex Engineering Systems: An Application to Space PSA," Operations Research, INFORMS, vol. 59(6), pages 1461-1476, December.
    43. Peter Klibanoff & Massimo Marinacci & Sujoy Mukerji, 2005. "A Smooth Model of Decision Making under Ambiguity," Econometrica, Econometric Society, vol. 73(6), pages 1849-1892, November.
    44. Filippi, Carlo, 2005. "A fresh view on the tolerance approach to sensitivity analysis in linear programming," European Journal of Operational Research, Elsevier, vol. 167(1), pages 1-19, November.
    45. Helton, Jon C. & Johnson, Jay D., 2011. "Quantification of margins and uncertainties: Alternative representations of epistemic uncertainty," Reliability Engineering and System Safety, Elsevier, vol. 96(9), pages 1034-1052.
    46. Mara, Thierry A. & Tarantola, Stefano, 2012. "Variance-based sensitivity indices for models with dependent inputs," Reliability Engineering and System Safety, Elsevier, vol. 107(C), pages 115-121.
    47. Jeremy E. Oakley & Anthony O'Hagan, 2004. "Probabilistic sensitivity analysis of complex models: a Bayesian approach," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 66(3), pages 751-769, August.
    48. van Groenendaal, W.J.H. & Kleijnen, J.P.C., 1997. "On the assessment of economic risk : Factorial design versus Monte Carlo methods," Other publications TiSEM fd2a2307-0812-4543-8151-7, Tilburg University, School of Economics and Management.
    49. Helton, J.C. & Johnson, J.D. & Sallaberry, C.J. & Storlie, C.B., 2006. "Survey of sampling-based methods for uncertainty and sensitivity analysis," Reliability Engineering and System Safety, Elsevier, vol. 91(10), pages 1175-1209.
    50. Liu, Ruixue & Owen, Art B., 2006. "Estimating Mean Dimensionality of Analysis of Variance Decompositions," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 712-721, June.
    51. Bettonvil, Bert & Kleijnen, Jack P. C., 1997. "Searching for important factors in simulation models with many factors: Sequential bifurcation," European Journal of Operational Research, Elsevier, vol. 96(1), pages 180-194, January.
    52. Mark Strong & Jeremy E. Oakley & Jim Chilcott, 2012. "Managing structural uncertainty in health economic decision models: a discrepancy approach," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 61(1), pages 25-45, January.
    53. Buzzard, Gregery T., 2012. "Global sensitivity analysis using sparse grid interpolation and polynomial chaos," Reliability Engineering and System Safety, Elsevier, vol. 107(C), pages 82-89.
    54. Storlie, Curtis B. & Reich, Brian J. & Helton, Jon C. & Swiler, Laura P. & Sallaberry, Cedric J., 2013. "Analysis of computationally demanding models with continuous and categorical inputs," Reliability Engineering and System Safety, Elsevier, vol. 113(C), pages 30-41.
    55. James C. Felli & Gordon B. Hazen, 2004. "Javelin Diagrams: A Graphical Tool for Probabilistic Sensitivity Analysis," Decision Analysis, INFORMS, vol. 1(2), pages 93-107, June.
    56. Helton, J.C. & Johnson, J.D. & Oberkampf, W.L. & Sallaberry, C.J., 2006. "Sensitivity analysis in conjunction with evidence theory representations of epistemic uncertainty," Reliability Engineering and System Safety, Elsevier, vol. 91(10), pages 1414-1434.
    57. Xiaoqun Wang, 2006. "On the Effects of Dimension Reduction Techniques on Some High-Dimensional Problems in Finance," Operations Research, INFORMS, vol. 54(6), pages 1063-1078, December.
    58. Hong Wan & Bruce E. Ankenman & Barry L. Nelson, 2006. "Controlled Sequential Bifurcation: A New Factor-Screening Method for Discrete-Event Simulation," Operations Research, INFORMS, vol. 54(4), pages 743-755, August.
    59. Plischke, Elmar, 2010. "An effective algorithm for computing global sensitivity indices (EASI)," Reliability Engineering and System Safety, Elsevier, vol. 95(4), pages 354-360.
    60. Gao, Xueli & Cui, Lirong & Li, Jinlin, 2007. "Analysis for joint importance of components in a coherent system," European Journal of Operational Research, Elsevier, vol. 182(1), pages 282-299, October.
    61. Hazen, Gordon B & Sounderpandian, Jayavel, 1999. "Lottery Acquisition versus Information Acquisition: Prices and Preference Reversals," Journal of Risk and Uncertainty, Springer, vol. 18(2), pages 125-136, August.
    62. Plischke, Elmar & Borgonovo, Emanuele & Smith, Curtis L., 2013. "Global sensitivity measures from given data," European Journal of Operational Research, Elsevier, vol. 226(3), pages 536-550.
    63. Borgonovo, E., 2010. "Sensitivity analysis with finite changes: An application to modified EOQ models," European Journal of Operational Research, Elsevier, vol. 200(1), pages 127-138, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mirko Ginocchi & Ferdinanda Ponci & Antonello Monti, 2021. "Sensitivity Analysis and Power Systems: Can We Bridge the Gap? A Review and a Guide to Getting Started," Energies, MDPI, vol. 14(24), pages 1-59, December.
    2. Emanuele Borgonovo & Gordon B. Hazen & Elmar Plischke, 2016. "A Common Rationale for Global Sensitivity Measures and Their Estimation," Risk Analysis, John Wiley & Sons, vol. 36(10), pages 1871-1895, October.
    3. Plischke, Elmar & Borgonovo, Emanuele & Smith, Curtis L., 2013. "Global sensitivity measures from given data," European Journal of Operational Research, Elsevier, vol. 226(3), pages 536-550.
    4. Wei, Pengfei & Lu, Zhenzhou & Song, Jingwen, 2015. "Variable importance analysis: A comprehensive review," Reliability Engineering and System Safety, Elsevier, vol. 142(C), pages 399-432.
    5. S. Cucurachi & E. Borgonovo & R. Heijungs, 2016. "A Protocol for the Global Sensitivity Analysis of Impact Assessment Models in Life Cycle Assessment," Risk Analysis, John Wiley & Sons, vol. 36(2), pages 357-377, February.
    6. Borgonovo, Emanuele & Buzzard, Gregery T. & Wendell, Richard E., 2018. "A global tolerance approach to sensitivity analysis in linear programming," European Journal of Operational Research, Elsevier, vol. 267(1), pages 321-337.
    7. Elmar Plischke & Emanuele Borgonovo, 2020. "Fighting the Curse of Sparsity: Probabilistic Sensitivity Measures From Cumulative Distribution Functions," Risk Analysis, John Wiley & Sons, vol. 40(12), pages 2639-2660, December.
    8. Tianyang Wang & James S. Dyer & Warren J. Hahn, 2017. "Sensitivity analysis of decision making under dependent uncertainties using copulas," EURO Journal on Decision Processes, Springer;EURO - The Association of European Operational Research Societies, vol. 5(1), pages 117-139, November.
    9. Borgonovo, Emanuele & Hazen, Gordon B. & Jose, Victor Richmond R. & Plischke, Elmar, 2021. "Probabilistic sensitivity measures as information value," European Journal of Operational Research, Elsevier, vol. 289(2), pages 595-610.
    10. Cheng, Lei & Lu, Zhenzhou & Zhang, Leigang, 2015. "Application of Rejection Sampling based methodology to variance based parametric sensitivity analysis," Reliability Engineering and System Safety, Elsevier, vol. 142(C), pages 9-18.
    11. E. Borgonovo & C. L. Smith, 2011. "A Study of Interactions in the Risk Assessment of Complex Engineering Systems: An Application to Space PSA," Operations Research, INFORMS, vol. 59(6), pages 1461-1476, December.
    12. Plischke, Elmar & Borgonovo, Emanuele, 2019. "Copula theory and probabilistic sensitivity analysis: Is there a connection?," European Journal of Operational Research, Elsevier, vol. 277(3), pages 1046-1059.
    13. Matieyendou Lamboni, 2020. "Uncertainty quantification: a minimum variance unbiased (joint) estimator of the non-normalized Sobol’ indices," Statistical Papers, Springer, vol. 61(5), pages 1939-1970, October.
    14. Xuefei Lu & Alessandro Rudi & Emanuele Borgonovo & Lorenzo Rosasco, 2020. "Faster Kriging: Facing High-Dimensional Simulators," Operations Research, INFORMS, vol. 68(1), pages 233-249, January.
    15. Awad, Mahmoud, 2017. "Analyzing sensitivity measures using moment-matching technique," Reliability Engineering and System Safety, Elsevier, vol. 159(C), pages 90-99.
    16. Barry Anderson & Emanuele Borgonovo & Marzio Galeotti & Roberto Roson, 2014. "Uncertainty in Climate Change Modeling: Can Global Sensitivity Analysis Be of Help?," Risk Analysis, John Wiley & Sons, vol. 34(2), pages 271-293, February.
    17. Emanuele Borgonovo & Marco Pangallo & Jan Rivkin & Leonardo Rizzo & Nicolaj Siggelkow, 2022. "Sensitivity analysis of agent-based models: a new protocol," Computational and Mathematical Organization Theory, Springer, vol. 28(1), pages 52-94, March.
    18. Matieyendou Lamboni, 2018. "Global sensitivity analysis: a generalized, unbiased and optimal estimator of total-effect variance," Statistical Papers, Springer, vol. 59(1), pages 361-386, March.
    19. Zhai, Qingqing & Yang, Jun & Xie, Min & Zhao, Yu, 2014. "Generalized moment-independent importance measures based on Minkowski distance," European Journal of Operational Research, Elsevier, vol. 239(2), pages 449-455.
    20. Borgonovo, E. & Peccati, L., 2011. "Finite change comparative statics for risk-coherent inventories," International Journal of Production Economics, Elsevier, vol. 131(1), pages 52-62, May.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:248:y:2016:i:3:p:869-887. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.