IDEAS home Printed from https://ideas.repec.org/a/wly/riskan/v22y2002i3p591-622.html
   My bibliography  Save this article

Illustration of Sampling‐Based Methods for Uncertainty and Sensitivity Analysis

Author

Listed:
  • J. C. Helton
  • F. J. Davis

Abstract

A sequence of linear, monotonic, and nonmonotonic test problems is used to illustrate sampling‐based uncertainty and sensitivity analysis procedures. Uncertainty results obtained with replicated random and Latin hypercube samples are compared, with the Latin hypercube samples tending to produce more stable results than the random samples. Sensitivity results obtained with the following procedures and/or measures are illustrated and compared: correlation coefficients (CCs), rank correlation coefficients (RCCs), common means (CMNs), common locations (CLs), common medians (CMDs), statistical independence (SI), standardized regression coefficients (SRCs), partial correlation coefficients (PCCs), standardized rank regression coefficients (SRRCs), partial rank correlation coefficients (PRCCs), stepwise regression analysis with raw and rank‐transformed data, and examination of scatter plots. The effectiveness of a given procedure and/or measure depends on the characteristics of the individual test problems, with (1) linear measures (i.e., CCs, PCCs, SRCs) performing well on the linear test problems, (2) measures based on rank transforms (i.e., RCCs, PRCCs, SRRCs) performing well on the monotonic test problems, and (3) measures predicated on searches for nonrandom patterns (i.e., CMNs, CLs, CMDs, SI) performing well on the nonmonotonic test problems.

Suggested Citation

  • J. C. Helton & F. J. Davis, 2002. "Illustration of Sampling‐Based Methods for Uncertainty and Sensitivity Analysis," Risk Analysis, John Wiley & Sons, vol. 22(3), pages 591-622, June.
    • Handle: RePEc:wly:riskan:v:22:y:2002:i:3:p:591-622
    DOI: 10.1111/0272-4332.00041
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/0272-4332.00041
    Download Restriction: no
    File URL: https://libkey.io/10.1111/0272-4332.00041?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Robert J. Budnitz & George Apostolakis & David M. Boore & Lloyd S. Cluff & Kevin J. Coppersmith & C. Allin Cornell & Peter A. Morris, 1998. "Use of Technical Expert Panels: Applications to Probabilistic Seismic Hazard Analysis," Risk Analysis, John Wiley & Sons, vol. 18(4), pages 463-469, August.
    2. F. Owen Hoffman & Jana S. Hammonds, 1994. "Propagation of Uncertainty in Risk Assessments: The Need to Distinguish Between Uncertainty Due to Lack of Knowledge and Uncertainty Due to Variability," Risk Analysis, John Wiley & Sons, vol. 14(5), pages 707-712, October.
    3. Stanley Kaplan & B. John Garrick, 1981. "On The Quantitative Definition of Risk," Risk Analysis, John Wiley & Sons, vol. 1(1), pages 11-27, March.
    4. Jon C. Helton, 1994. "Treatment of Uncertainty in Performance Assessments for Complex Systems," Risk Analysis, John Wiley & Sons, vol. 14(4), pages 483-511, August.
    5. Thomas E. McKone, 1994. "Uncertainty and Variability in Human Exposures to Soil Contaminants Through Home‐Grown Food: A Monte Carlo Assessment," Risk Analysis, John Wiley & Sons, vol. 14(4), pages 449-463, August.
    6. Saltelli, A. & Andres, T. H. & Homma, T., 1993. "Sensitivity analysis of model output : An investigation of new techniques," Computational Statistics & Data Analysis, Elsevier, vol. 15(2), pages 211-238, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sakurahara, Tatsuya & Schumock, Grant & Reihani, Seyed & Kee, Ernie & Mohaghegh, Zahra, 2019. "Simulation-Informed Probabilistic Methodology for Common Cause Failure Analysis," Reliability Engineering and System Safety, Elsevier, vol. 185(C), pages 84-99.
    2. Isadora Antoniano‐Villalobos & Emanuele Borgonovo & Sumeda Siriwardena, 2018. "Which Parameters Are Important? Differential Importance Under Uncertainty," Risk Analysis, John Wiley & Sons, vol. 38(11), pages 2459-2477, November.
    3. S. Cucurachi & E. Borgonovo & R. Heijungs, 2016. "A Protocol for the Global Sensitivity Analysis of Impact Assessment Models in Life Cycle Assessment," Risk Analysis, John Wiley & Sons, vol. 36(2), pages 357-377, February.
    4. Elmar Plischke & Emanuele Borgonovo, 2020. "Fighting the Curse of Sparsity: Probabilistic Sensitivity Measures From Cumulative Distribution Functions," Risk Analysis, John Wiley & Sons, vol. 40(12), pages 2639-2660, December.
    5. Frank H. Koch & Denys Yemshanov & Daniel W. McKenney & William D. Smith, 2009. "Evaluating Critical Uncertainty Thresholds in a Spatial Model of Forest Pest Invasion Risk," Risk Analysis, John Wiley & Sons, vol. 29(9), pages 1227-1241, September.
    6. Terje Aven, 2020. "Risk Science Contributions: Three Illustrating Examples," Risk Analysis, John Wiley & Sons, vol. 40(10), pages 1889-1899, October.
    7. Zhou, Changcong & Zhang, Hanlin & Valdebenito, Marcos A. & Zhao, Haodong, 2022. "A general hierarchical ensemble-learning framework for structural reliability analysis," Reliability Engineering and System Safety, Elsevier, vol. 225(C).
    8. Emanuele Borgonovo & Gordon B. Hazen & Elmar Plischke, 2016. "A Common Rationale for Global Sensitivity Measures and Their Estimation," Risk Analysis, John Wiley & Sons, vol. 36(10), pages 1871-1895, October.
    9. Barry Anderson & Emanuele Borgonovo & Marzio Galeotti & Roberto Roson, 2014. "Uncertainty in Climate Change Modeling: Can Global Sensitivity Analysis Be of Help?," Risk Analysis, John Wiley & Sons, vol. 34(2), pages 271-293, February.
    10. H. Christopher Frey, 2002. "Introduction to Special Section on Sensitivity Analysis and Summary of NCSU/USDA Workshop on Sensitivity Analysis," Risk Analysis, John Wiley & Sons, vol. 22(3), pages 539-545, June.
    11. Emanuele Borgonovo, 2008. "Epistemic Uncertainty in the Ranking and Categorization of Probabilistic Safety Assessment Model Elements: Issues and Findings," Risk Analysis, John Wiley & Sons, vol. 28(4), pages 983-1001, August.
    12. Andreas Tsanakas & Pietro Millossovich, 2016. "Sensitivity Analysis Using Risk Measures," Risk Analysis, John Wiley & Sons, vol. 36(1), pages 30-48, January.
    13. Jared M. Hansen & Paul Wilson, 2023. "Modeling and explaining the growth patterns over time of country-specific website clickstream metrics," Journal of Marketing Analytics, Palgrave Macmillan, vol. 11(4), pages 561-576, December.
    14. Alastair Heffernan & Ella Barber & Ranjeeta Thomas & Christophe Fraser & Michael Pickles & Anne Cori, 2016. "Impact and Cost-Effectiveness of Point-Of-Care CD4 Testing on the HIV Epidemic in South Africa," PLOS ONE, Public Library of Science, vol. 11(7), pages 1-12, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kimberly M. Thompson, 2002. "Variability and Uncertainty Meet Risk Management and Risk Communication," Risk Analysis, John Wiley & Sons, vol. 22(3), pages 647-654, June.
    2. Helton, Jon C. & Brooks, Dusty M. & Sallaberry, Cédric J., 2020. "Margins associated with loss of assured safety for systems with multiple weak links and strong links," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    3. Emanuele Borgonovo, 2008. "Epistemic Uncertainty in the Ranking and Categorization of Probabilistic Safety Assessment Model Elements: Issues and Findings," Risk Analysis, John Wiley & Sons, vol. 28(4), pages 983-1001, August.
    4. Amirhossein Mokhtari & H. Christopher Frey, 2005. "Sensitivity Analysis of a Two‐Dimensional Probabilistic Risk Assessment Model Using Analysis of Variance," Risk Analysis, John Wiley & Sons, vol. 25(6), pages 1511-1529, December.
    5. S. Cucurachi & E. Borgonovo & R. Heijungs, 2016. "A Protocol for the Global Sensitivity Analysis of Impact Assessment Models in Life Cycle Assessment," Risk Analysis, John Wiley & Sons, vol. 36(2), pages 357-377, February.
    6. Helton, Jon C. & Brooks, Dusty M. & Sallaberry, Cédric J., 2020. "Property values associated with the failure of individual links in a system with multiple weak and strong links," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    7. Aven, Terje, 2020. "Three influential risk foundation papers from the 80s and 90s: Are they still state-of-the-art?," Reliability Engineering and System Safety, Elsevier, vol. 193(C).
    8. Julia J. Pet‐Armacost & Jose Sepulveda & Milton Sakude, 1999. "Monte Carlo Sensitivity Analysis of Unknown Parameters in Hazardous Materials Transportation Risk Assessment," Risk Analysis, John Wiley & Sons, vol. 19(6), pages 1173-1184, December.
    9. Thomas Ying‐Jeh Chen & Valerie Nicole Washington & Terje Aven & Seth David Guikema, 2020. "Review and Evaluation of the J100‐10 Risk and Resilience Management Standard for Water and Wastewater Systems," Risk Analysis, John Wiley & Sons, vol. 40(3), pages 608-623, March.
    10. Andrea Saltelli & Stefano Tarantola & Karen Chad, 1998. "Presenting Results from Model Based Studies to Decision‐Makers: Can Sensitivity Analysis Be a Defogging Agent?," Risk Analysis, John Wiley & Sons, vol. 18(6), pages 799-803, December.
    11. Sarazin, Gabriel & Morio, Jérôme & Lagnoux, Agnès & Balesdent, Mathieu & Brevault, Loïc, 2021. "Reliability-oriented sensitivity analysis in presence of data-driven epistemic uncertainty," Reliability Engineering and System Safety, Elsevier, vol. 215(C).
    12. Junyu Zheng & H. Christopher Frey, 2004. "Quantification of Variability and Uncertainty Using Mixture Distributions: Evaluation of Sample Size, Mixing Weights, and Separation Between Components," Risk Analysis, John Wiley & Sons, vol. 24(3), pages 553-571, June.
    13. George E. Apostolakis, 2004. "How Useful Is Quantitative Risk Assessment?," Risk Analysis, John Wiley & Sons, vol. 24(3), pages 515-520, June.
    14. Victor R. Vasquez & Wallace B. Whiting, 2005. "Accounting for Both Random Errors and Systematic Errors in Uncertainty Propagation Analysis of Computer Models Involving Experimental Measurements with Monte Carlo Methods," Risk Analysis, John Wiley & Sons, vol. 25(6), pages 1669-1681, December.
    15. M. Elisabeth Paté‐Cornell, 1999. "Conditional Uncertainty Analysis and Implications for Decision Making: The Case of WIPP," Risk Analysis, John Wiley & Sons, vol. 19(5), pages 995-1002, October.
    16. Jon C. Helton & William L. Oberkampf & Jay D. Johnson, 2005. "Competing Failure Risk Analysis Using Evidence Theory," Risk Analysis, John Wiley & Sons, vol. 25(4), pages 973-995, August.
    17. Paul S. Price & Steave H. Su & Jeff R. Harrington & Russell E. Keenan, 1996. "Uncertainty and Variation in Indirect Exposure Assessments: An Analysis of Exposure to Tetrachlorodibenzo‐p‐Dioxin from a Beef Consumption Pathway," Risk Analysis, John Wiley & Sons, vol. 16(2), pages 263-277, April.
    18. J. C. Helton & D. R. Anderson & H.‐N. Jow & M. G. Marietta & G. Basabilvazo, 1999. "Performance Assessment in Support of the 1996 Compliance Certification Application for the Waste Isolation Pilot Plant," Risk Analysis, John Wiley & Sons, vol. 19(5), pages 959-986, October.
    19. H. Christopher Frey & David E. Burmaster, 1999. "Methods for Characterizing Variability and Uncertainty: Comparison of Bootstrap Simulation and Likelihood‐Based Approaches," Risk Analysis, John Wiley & Sons, vol. 19(1), pages 109-130, February.
    20. Gundula Glowka & Andreas Kallmünzer & Anita Zehrer, 2021. "Enterprise risk management in small and medium family enterprises: the role of family involvement and CEO tenure," International Entrepreneurship and Management Journal, Springer, vol. 17(3), pages 1213-1231, September.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:riskan:v:22:y:2002:i:3:p:591-622. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1111/(ISSN)1539-6924 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.