IDEAS home Printed from https://ideas.repec.org/a/eee/reensy/v142y2015icp399-432.html
   My bibliography  Save this article

Variable importance analysis: A comprehensive review

Author

Listed:
  • Wei, Pengfei
  • Lu, Zhenzhou
  • Song, Jingwen

Abstract

Measuring variable importance for computational models or measured data is an important task in many applications. It has drawn our attention that the variable importance analysis (VIA) techniques were developed independently in many disciplines. We are strongly aware of the necessity to aggregate all the good practices in each discipline, and compare the relative merits of each method, so as to instruct the practitioners to choose the optimal methods to meet different analysis purposes, and to guide current research on VIA. To this end, all the good practices, including seven groups of methods, i.e., the difference-based variable importance measures (VIMs), parametric regression and related VIMs, nonparametric regression techniques, hypothesis test techniques, variance-based VIMs, moment-independent VIMs and graphic VIMs, are reviewed and compared with a numerical test example set in two situations (independent and dependent cases). For ease of use, the recommendations are provided for different types of applications, and packages as well as software for implementing these VIA techniques are collected. Prospects for future study of VIA techniques are also proposed.

Suggested Citation

  • Wei, Pengfei & Lu, Zhenzhou & Song, Jingwen, 2015. "Variable importance analysis: A comprehensive review," Reliability Engineering and System Safety, Elsevier, vol. 142(C), pages 399-432.
    • Handle: RePEc:eee:reensy:v:142:y:2015:i:c:p:399-432
    DOI: 10.1016/j.ress.2015.05.018
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0951832015001672
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ress.2015.05.018?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Kucherenko, S. & Rodriguez-Fernandez, M. & Pantelides, C. & Shah, N., 2009. "Monte Carlo evaluation of derivative-based global sensitivity measures," Reliability Engineering and System Safety, Elsevier, vol. 94(7), pages 1135-1148.
    2. Archer, Kellie J. & Kimes, Ryan V., 2008. "Empirical characterization of random forest variable importance measures," Computational Statistics & Data Analysis, Elsevier, vol. 52(4), pages 2249-2260, January.
    3. Kleijnen, Jack P.C., 2009. "Kriging metamodeling in simulation: A review," European Journal of Operational Research, Elsevier, vol. 192(3), pages 707-716, February.
    4. S. N. Wood, 2000. "Modelling and smoothing parameter estimation with multiple quadratic penalties," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 62(2), pages 413-428.
    5. Campbell, Katherine & McKay, Michael D. & Williams, Brian J., 2006. "Sensitivity analysis when model outputs are functions," Reliability Engineering and System Safety, Elsevier, vol. 91(10), pages 1468-1472.
    6. Tarantola, S. & Kopustinskas, V. & Bolado-Lavin, R. & Kaliatka, A. & Ušpuras, E. & Vaišnoras, M., 2012. "Sensitivity analysis using contribution to sample variance plot: Application to a water hammer model," Reliability Engineering and System Safety, Elsevier, vol. 99(C), pages 62-73.
    7. Storlie, Curtis B. & Helton, Jon C., 2008. "Multiple predictor smoothing methods for sensitivity analysis: Description of techniques," Reliability Engineering and System Safety, Elsevier, vol. 93(1), pages 28-54.
    8. Clive Granger & Jin‐Lung Lin, 1994. "Using The Mutual Information Coefficient To Identify Lags In Nonlinear Models," Journal of Time Series Analysis, Wiley Blackwell, vol. 15(4), pages 371-384, July.
    9. Bolado-Lavin, R. & Castaings, W. & Tarantola, S., 2009. "Contribution to the sample mean plot for graphical and numerical sensitivity analysis," Reliability Engineering and System Safety, Elsevier, vol. 94(6), pages 1041-1049.
    10. Sudret, Bruno, 2008. "Global sensitivity analysis using polynomial chaos expansions," Reliability Engineering and System Safety, Elsevier, vol. 93(7), pages 964-979.
    11. Lamboni, M. & Iooss, B. & Popelin, A.-L. & Gamboa, F., 2013. "Derivative-based global sensitivity measures: General links with Sobol’ indices and numerical tests," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 87(C), pages 45-54.
    12. Mara, Thierry A. & Tarantola, Stefano, 2012. "Variance-based sensitivity indices for models with dependent inputs," Reliability Engineering and System Safety, Elsevier, vol. 107(C), pages 115-121.
    13. Lamboni, Matieyendou & Monod, Hervé & Makowski, David, 2011. "Multivariate sensitivity analysis to measure global contribution of input factors in dynamic models," Reliability Engineering and System Safety, Elsevier, vol. 96(4), pages 450-459.
    14. Xu, C. & Gertner, G., 2007. "Extending a global sensitivity analysis technique to models with correlated parameters," Computational Statistics & Data Analysis, Elsevier, vol. 51(12), pages 5579-5590, August.
    15. Shih, Yu-Shan & Tsai, Hsin-Wen, 2004. "Variable selection bias in regression trees with constant fits," Computational Statistics & Data Analysis, Elsevier, vol. 45(3), pages 595-607, April.
    16. Storlie, Curtis B. & Reich, Brian J. & Helton, Jon C. & Swiler, Laura P. & Sallaberry, Cedric J., 2013. "Analysis of computationally demanding models with continuous and categorical inputs," Reliability Engineering and System Safety, Elsevier, vol. 113(C), pages 30-41.
    17. Wei, Pengfei & Lu, Zhenzhou & Yuan, Xiukai, 2013. "Monte Carlo simulation for moment-independent sensitivity analysis," Reliability Engineering and System Safety, Elsevier, vol. 110(C), pages 60-67.
    18. Storlie, Curtis B. & Helton, Jon C., 2008. "Multiple predictor smoothing methods for sensitivity analysis: Example results," Reliability Engineering and System Safety, Elsevier, vol. 93(1), pages 55-77.
    19. Sobol’, I.M. & Tarantola, S. & Gatelli, D. & Kucherenko, S.S. & Mauntz, W., 2007. "Estimating the approximation error when fixing unessential factors in global sensitivity analysis," Reliability Engineering and System Safety, Elsevier, vol. 92(7), pages 957-960.
    20. Kucherenko, Sergei & Feil, Balazs & Shah, Nilay & Mauntz, Wolfgang, 2011. "The identification of model effective dimensions using global sensitivity analysis," Reliability Engineering and System Safety, Elsevier, vol. 96(4), pages 440-449.
    21. Goldstein Benjamin A & Polley Eric C & Briggs Farren B. S., 2011. "Random Forests for Genetic Association Studies," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 10(1), pages 1-34, July.
    22. Saltelli, Andrea & Campolongo, Francesca & Cariboni, Jessica, 2009. "Screening important inputs in models with strong interaction properties," Reliability Engineering and System Safety, Elsevier, vol. 94(7), pages 1149-1155.
    23. Borgonovo, E., 2010. "Sensitivity analysis with finite changes: An application to modified EOQ models," European Journal of Operational Research, Elsevier, vol. 200(1), pages 127-138, January.
    24. Borgonovo, E., 2007. "Differential, criticality and Birnbaum importance measures: An application to basic event, groups and SSCs in event trees and binary decision diagrams," Reliability Engineering and System Safety, Elsevier, vol. 92(10), pages 1458-1467.
    25. Sobol’, I.M. & Kucherenko, S., 2009. "Derivative based global sensitivity measures and their link with global sensitivity indices," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 79(10), pages 3009-3017.
    26. Helton, J.C. & Hansen, C.W. & Sallaberry, C.J., 2014. "Expected dose for the nominal scenario class in the 2008 performance assessment for the proposed high-level radioactive waste repository at Yucca Mountain, Nevada," Reliability Engineering and System Safety, Elsevier, vol. 122(C), pages 267-271.
    27. Xu, Chonggang & Gertner, George Zdzislaw, 2008. "Uncertainty and sensitivity analysis for models with correlated parameters," Reliability Engineering and System Safety, Elsevier, vol. 93(10), pages 1563-1573.
    28. Borgonovo, E., 2007. "A new uncertainty importance measure," Reliability Engineering and System Safety, Elsevier, vol. 92(6), pages 771-784.
    29. Garcia-Cabrejo, Oscar & Valocchi, Albert, 2014. "Global Sensitivity Analysis for multivariate output using Polynomial Chaos Expansion," Reliability Engineering and System Safety, Elsevier, vol. 126(C), pages 25-36.
    30. Storlie, Curtis B. & Swiler, Laura P. & Helton, Jon C. & Sallaberry, Cedric J., 2009. "Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models," Reliability Engineering and System Safety, Elsevier, vol. 94(11), pages 1735-1763.
    31. E. Borgonovo & S. Tarantola & E. Plischke & M. D. Morris, 2014. "Transformations and invariance in the sensitivity analysis of computer experiments," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 76(5), pages 925-947, November.
    32. Xu, Chonggang & Gertner, George Zdzislaw, 2008. "A general first-order global sensitivity analysis method," Reliability Engineering and System Safety, Elsevier, vol. 93(7), pages 1060-1071.
    33. Hao, Wenrui & Lu, Zhenzhou & Tian, Longfei, 2012. "Importance measure of correlated normal variables and its sensitivity analysis," Reliability Engineering and System Safety, Elsevier, vol. 99(C), pages 151-160.
    34. Ratto, M. & Pagano, A. & Young, P.C., 2009. "Non-parametric estimation of conditional moments for sensitivity analysis," Reliability Engineering and System Safety, Elsevier, vol. 94(2), pages 237-243.
    35. Zuber Verena & Strimmer Korbinian, 2011. "High-Dimensional Regression and Variable Selection Using CAR Scores," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 10(1), pages 1-27, July.
    36. Tarantola, S. & Gatelli, D. & Mara, T.A., 2006. "Random balance designs for the estimation of first order global sensitivity indices," Reliability Engineering and System Safety, Elsevier, vol. 91(6), pages 717-727.
    37. Tian, Wei, 2013. "A review of sensitivity analysis methods in building energy analysis," Renewable and Sustainable Energy Reviews, Elsevier, vol. 20(C), pages 411-419.
    38. Manel Baucells & Emanuele Borgonovo, 2013. "Invariant Probabilistic Sensitivity Analysis," Management Science, INFORMS, vol. 59(11), pages 2536-2549, November.
    39. Jeremy E. Oakley & Anthony O'Hagan, 2004. "Probabilistic sensitivity analysis of complex models: a Bayesian approach," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 66(3), pages 751-769, August.
    40. Helton, J.C. & Johnson, J.D. & Sallaberry, C.J. & Storlie, C.B., 2006. "Survey of sampling-based methods for uncertainty and sensitivity analysis," Reliability Engineering and System Safety, Elsevier, vol. 91(10), pages 1175-1209.
    41. Mara, Thierry Alex, 2009. "Extension of the RBD-FAST method to the computation of global sensitivity indices," Reliability Engineering and System Safety, Elsevier, vol. 94(8), pages 1274-1281.
    42. Rahman, Sharif, 2011. "Global sensitivity analysis by polynomial dimensional decomposition," Reliability Engineering and System Safety, Elsevier, vol. 96(7), pages 825-837.
    43. Cao, Jiaokun & Du, Farong & Ding, Shuiting, 2013. "Global sensitivity analysis for dynamic systems with stochastic input processes," Reliability Engineering and System Safety, Elsevier, vol. 118(C), pages 106-117.
    44. Helton, J.C. & Hansen, C.W. & Sallaberry, C.J., 2014. "Conceptual structure and computational organization of the 2008 performance assessment for the proposed high-level radioactive waste repository at Yucca Mountain, Nevada," Reliability Engineering and System Safety, Elsevier, vol. 122(C), pages 223-248.
    45. Xu, Chonggang & Gertner, George, 2011. "Understanding and comparisons of different sampling approaches for the Fourier Amplitudes Sensitivity Test (FAST)," Computational Statistics & Data Analysis, Elsevier, vol. 55(1), pages 184-198, January.
    46. Wei, Pengfei & Lu, Zhenzhou & Ruan, Wenbin & Song, Jingwen, 2014. "Regional sensitivity analysis using revised mean and variance ratio functions," Reliability Engineering and System Safety, Elsevier, vol. 121(C), pages 121-135.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Cheng, Lei & Lu, Zhenzhou & Zhang, Leigang, 2015. "Application of Rejection Sampling based methodology to variance based parametric sensitivity analysis," Reliability Engineering and System Safety, Elsevier, vol. 142(C), pages 9-18.
    2. Borgonovo, Emanuele & Plischke, Elmar, 2016. "Sensitivity analysis: A review of recent advances," European Journal of Operational Research, Elsevier, vol. 248(3), pages 869-887.
    3. Mirko Ginocchi & Ferdinanda Ponci & Antonello Monti, 2021. "Sensitivity Analysis and Power Systems: Can We Bridge the Gap? A Review and a Guide to Getting Started," Energies, MDPI, vol. 14(24), pages 1-59, December.
    4. Matieyendou Lamboni, 2020. "Uncertainty quantification: a minimum variance unbiased (joint) estimator of the non-normalized Sobol’ indices," Statistical Papers, Springer, vol. 61(5), pages 1939-1970, October.
    5. Matieyendou Lamboni, 2018. "Global sensitivity analysis: a generalized, unbiased and optimal estimator of total-effect variance," Statistical Papers, Springer, vol. 59(1), pages 361-386, March.
    6. Wei, Pengfei & Lu, Zhenzhou & Song, Jingwen, 2015. "Regional and parametric sensitivity analysis of Sobol׳ indices," Reliability Engineering and System Safety, Elsevier, vol. 137(C), pages 87-100.
    7. Chen, Xin & Molina-Cristóbal, Arturo & Guenov, Marin D. & Riaz, Atif, 2019. "Efficient method for variance-based sensitivity analysis," Reliability Engineering and System Safety, Elsevier, vol. 181(C), pages 97-115.
    8. Sinan Xiao & Zhenzhou Lu & Pan Wang, 2018. "Multivariate Global Sensitivity Analysis Based on Distance Components Decomposition," Risk Analysis, John Wiley & Sons, vol. 38(12), pages 2703-2721, December.
    9. Ge, Qiao & Menendez, Monica, 2017. "Extending Morris method for qualitative global sensitivity analysis of models with dependent inputs," Reliability Engineering and System Safety, Elsevier, vol. 162(C), pages 28-39.
    10. Mara, Thierry A. & Tarantola, Stefano, 2012. "Variance-based sensitivity indices for models with dependent inputs," Reliability Engineering and System Safety, Elsevier, vol. 107(C), pages 115-121.
    11. Di Maio, Francesco & Bandini, Alessandro & Zio, Enrico & Alberola, Sofia Carlos & Sanchez-Saez, Francisco & Martorell, Sebastián, 2016. "Bootstrapped-ensemble-based Sensitivity Analysis of a trace thermal-hydraulic model based on a limited number of PWR large break loca simulations," Reliability Engineering and System Safety, Elsevier, vol. 153(C), pages 122-134.
    12. Wei, Pengfei & Lu, Zhenzhou & Ruan, Wenbin & Song, Jingwen, 2014. "Regional sensitivity analysis using revised mean and variance ratio functions," Reliability Engineering and System Safety, Elsevier, vol. 121(C), pages 121-135.
    13. Konakli, Katerina & Sudret, Bruno, 2016. "Global sensitivity analysis using low-rank tensor approximations," Reliability Engineering and System Safety, Elsevier, vol. 156(C), pages 64-83.
    14. Plischke, Elmar, 2012. "An adaptive correlation ratio method using the cumulative sum of the reordered output," Reliability Engineering and System Safety, Elsevier, vol. 107(C), pages 149-156.
    15. Tatsuya Sakurahara & Seyed Reihani & Ernie Kee & Zahra Mohaghegh, 2020. "Global importance measure methodology for integrated probabilistic risk assessment," Journal of Risk and Reliability, , vol. 234(2), pages 377-396, April.
    16. Buzzard, Gregery T., 2012. "Global sensitivity analysis using sparse grid interpolation and polynomial chaos," Reliability Engineering and System Safety, Elsevier, vol. 107(C), pages 82-89.
    17. Xiao, Sinan & Lu, Zhenzhou & Wang, Pan, 2018. "Multivariate global sensitivity analysis for dynamic models based on wavelet analysis," Reliability Engineering and System Safety, Elsevier, vol. 170(C), pages 20-30.
    18. Zhai, Qingqing & Yang, Jun & Zhao, Yu, 2014. "Space-partition method for the variance-based sensitivity analysis: Optimal partition scheme and comparative study," Reliability Engineering and System Safety, Elsevier, vol. 131(C), pages 66-82.
    19. Hao, Wenrui & Lu, Zhenzhou & Wei, Pengfei, 2013. "Uncertainty importance measure for models with correlated normal variables," Reliability Engineering and System Safety, Elsevier, vol. 112(C), pages 48-58.
    20. Wu, Zeping & Wang, Donghui & Okolo N, Patrick & Hu, Fan & Zhang, Weihua, 2016. "Global sensitivity analysis using a Gaussian Radial Basis Function metamodel," Reliability Engineering and System Safety, Elsevier, vol. 154(C), pages 171-179.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:reensy:v:142:y:2015:i:c:p:399-432. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/reliability-engineering-and-system-safety .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.