IDEAS home Printed from https://ideas.repec.org/a/eee/reensy/v254y2025ipbs0951832024006173.html
   My bibliography  Save this article

A sensitivity analysis based trade-off between probabilistic model identification and statistical estimation

Author

Listed:
  • Surget, Charles
  • Dubreuil, Sylvain
  • Morio, Jérôme
  • Mattrand, Cécile
  • Bourinet, Jean-Marc
  • Gayton, Nicolas

Abstract

In a context of uncertainty quantification, the probabilistic model of a random vector at the input of a computational code is not always known. An identification of the joint distribution on a restricted sample of experimental data can lead to a bad calibration of the model. The quantity of interest estimated at the output of the code is then subject to a bi-level epistemic uncertainty that must be properly quantified. A first level arises from the statistical estimation whilst a second one comes from the identification of the probabilistic model. Each epistemic uncertainty can thus be reduced by an enrichment with new data, either by increasing the size of the estimation sample or by increasing the size of the identification sample. When gathering data is costly, it is then interesting to know which uncertainty source to reduce first, thus introducing a trade-off between simulation and physical experiment. This paper aims at presenting a sensitivity-analysis-guided enrichment procedure in a small data context to improve the estimation quality of a quantity of interest. The proposed methodology is shown to be both low cost and adaptive by introducing importance-sampling-based methods. The performance of the guided enrichment procedure is assessed on three examples.

Suggested Citation

  • Surget, Charles & Dubreuil, Sylvain & Morio, Jérôme & Mattrand, Cécile & Bourinet, Jean-Marc & Gayton, Nicolas, 2025. "A sensitivity analysis based trade-off between probabilistic model identification and statistical estimation," Reliability Engineering and System Safety, Elsevier, vol. 254(PB).
  • Handle: RePEc:eee:reensy:v:254:y:2025:i:pb:s0951832024006173
    DOI: 10.1016/j.ress.2024.110545
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0951832024006173
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ress.2024.110545?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Saltelli, Andrea & Ratto, Marco & Tarantola, Stefano & Campolongo, Francesca, 2006. "Sensitivity analysis practices: Strategies for model-based inference," Reliability Engineering and System Safety, Elsevier, vol. 91(10), pages 1109-1125.
    2. Sarazin, Gabriel & Morio, Jérôme & Lagnoux, Agnès & Balesdent, Mathieu & Brevault, Loïc, 2021. "Reliability-oriented sensitivity analysis in presence of data-driven epistemic uncertainty," Reliability Engineering and System Safety, Elsevier, vol. 215(C).
    3. Mara, Thierry A. & Tarantola, Stefano, 2012. "Variance-based sensitivity indices for models with dependent inputs," Reliability Engineering and System Safety, Elsevier, vol. 107(C), pages 115-121.
    4. Daniel Ackerberg, 2009. "A new use of importance sampling to reduce computational burden in simulation estimation," Quantitative Marketing and Economics (QME), Springer, vol. 7(4), pages 343-376, December.
    5. Jung, Yongsu & Lee, Ikjin, 2021. "Optimal design of experiments for optimization-based model calibration using Fisher information matrix," Reliability Engineering and System Safety, Elsevier, vol. 216(C).
    6. Nannapaneni, Saideep & Mahadevan, Sankaran, 2016. "Reliability analysis under epistemic uncertainty," Reliability Engineering and System Safety, Elsevier, vol. 155(C), pages 9-20.
    7. Nils-Bastian Heidenreich & Anja Schindler & Stefan Sperlich, 2013. "Bandwidth selection for kernel density estimation: a review of fully automatic selectors," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 97(4), pages 403-433, October.
    8. Sankararaman, S. & Mahadevan, S., 2013. "Separating the contributions of variability and parameter uncertainty in probability distributions," Reliability Engineering and System Safety, Elsevier, vol. 112(C), pages 187-199.
    9. Chabridon, Vincent & Balesdent, Mathieu & Bourinet, Jean-Marc & Morio, Jérôme & Gayton, Nicolas, 2018. "Reliability-based sensitivity estimators of rare event probability in the presence of distribution parameter uncertainty," Reliability Engineering and System Safety, Elsevier, vol. 178(C), pages 164-178.
    10. Li, Genyuan & Rabitz, Herschel, 2017. "Relationship between sensitivity indices defined by variance- and covariance-based methods," Reliability Engineering and System Safety, Elsevier, vol. 167(C), pages 136-157.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wang, Zhenqiang & Jia, Gaofeng, 2020. "Augmented sample-based approach for efficient evaluation of risk sensitivity with respect to epistemic uncertainty in distribution parameters," Reliability Engineering and System Safety, Elsevier, vol. 197(C).
    2. McFarland, John & DeCarlo, Erin, 2020. "A Monte Carlo framework for probabilistic analysis and variance decomposition with distribution parameter uncertainty," Reliability Engineering and System Safety, Elsevier, vol. 197(C).
    3. Zhang, Xiaodong & Dimitrov, Nikolay, 2024. "Variable importance analysis of wind turbine extreme responses with Shapley value explanation," Renewable Energy, Elsevier, vol. 232(C).
    4. Tabandeh, Armin & Sharma, Neetesh & Gardoni, Paolo, 2022. "Uncertainty propagation in risk and resilience analysis of hierarchical systems," Reliability Engineering and System Safety, Elsevier, vol. 219(C).
    5. Ajenjo, Antoine & Ardillon, Emmanuel & Chabridon, Vincent & Cogan, Scott & Sadoulet-Reboul, Emeline, 2023. "Robustness evaluation of the reliability of penstocks combining line sampling and neural networks," Reliability Engineering and System Safety, Elsevier, vol. 234(C).
    6. Xiang Peng & Xiaoqing Xu & Jiquan Li & Shaofei Jiang, 2021. "A Sampling-Based Sensitivity Analysis Method Considering the Uncertainties of Input Variables and Their Distribution Parameters," Mathematics, MDPI, vol. 9(10), pages 1-18, May.
    7. Kapusuzoglu, Berkcan & Mahadevan, Sankaran, 2021. "Information fusion and machine learning for sensitivity analysis using physics knowledge and experimental data," Reliability Engineering and System Safety, Elsevier, vol. 214(C).
    8. Straub, Daniel & Ehre, Max & Papaioannou, Iason, 2022. "Decision-theoretic reliability sensitivity," Reliability Engineering and System Safety, Elsevier, vol. 221(C).
    9. Mara, Thierry A. & Becker, William E., 2021. "Polynomial chaos expansion for sensitivity analysis of model output with dependent inputs," Reliability Engineering and System Safety, Elsevier, vol. 214(C).
    10. Xie, Xiangzhong & Schenkendorf, René & Krewer, Ulrike, 2019. "Efficient sensitivity analysis and interpretation of parameter correlations in chemical engineering," Reliability Engineering and System Safety, Elsevier, vol. 187(C), pages 159-173.
    11. Jung, Yongsu & Jo, Hwisang & Choo, Jeonghwan & Lee, Ikjin, 2022. "Statistical model calibration and design optimization under aleatory and epistemic uncertainty," Reliability Engineering and System Safety, Elsevier, vol. 222(C).
    12. Imron, Muhammad Ali & Gergs, Andre & Berger, Uta, 2012. "Structure and sensitivity analysis of individual-based predator–prey models," Reliability Engineering and System Safety, Elsevier, vol. 107(C), pages 71-81.
    13. Li, Chenzhao & Mahadevan, Sankaran, 2018. "Efficient approximate inference in Bayesian networks with continuous variables," Reliability Engineering and System Safety, Elsevier, vol. 169(C), pages 269-280.
    14. Izquierdo, J. & Márquez, A. Crespo & Uribetxebarria, J. & Erguido, A., 2020. "On the importance of assessing the operational context impact on maintenance management for life cycle cost of wind energy projects," Renewable Energy, Elsevier, vol. 153(C), pages 1100-1110.
    15. Ertian Chen, 2025. "Model-Adaptive Approach to Dynamic Discrete Choice Models with Large State Spaces," Papers 2501.18746, arXiv.org, revised Jun 2025.
    16. Andrew Ching & Susumu Imai & Masakazu Ishihara & Neelam Jain, 2012. "A practitioner’s guide to Bayesian estimation of discrete choice dynamic programming models," Quantitative Marketing and Economics (QME), Springer, vol. 10(2), pages 151-196, June.
    17. Lu, Xuefei & Borgonovo, Emanuele, 2023. "Global sensitivity analysis in epidemiological modeling," European Journal of Operational Research, Elsevier, vol. 304(1), pages 9-24.
    18. Tianyang Wang & James S. Dyer & Warren J. Hahn, 2017. "Sensitivity analysis of decision making under dependent uncertainties using copulas," EURO Journal on Decision Processes, Springer;EURO - The Association of European Operational Research Societies, vol. 5(1), pages 117-139, November.
    19. James W. Roberts & Andrew Sweeting, 2013. "When Should Sellers Use Auctions?," American Economic Review, American Economic Association, vol. 103(5), pages 1830-1861, August.
    20. Drignei, Dorin, 2011. "A general statistical model for computer experiments with time series output," Reliability Engineering and System Safety, Elsevier, vol. 96(4), pages 460-467.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:reensy:v:254:y:2025:i:pb:s0951832024006173. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/reliability-engineering-and-system-safety .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.