IDEAS home Printed from https://ideas.repec.org/a/wly/riskan/v25y2005i6p1669-1681.html
   My bibliography  Save this article

Accounting for Both Random Errors and Systematic Errors in Uncertainty Propagation Analysis of Computer Models Involving Experimental Measurements with Monte Carlo Methods

Author

Listed:
  • Victor R. Vasquez
  • Wallace B. Whiting

Abstract

A Monte Carlo method is presented to study the effect of systematic and random errors on computer models mainly dealing with experimental data. It is a common assumption in this type of models (linear and nonlinear regression, and nonregression computer models) involving experimental measurements that the error sources are mainly random and independent with no constant background errors (systematic errors). However, from comparisons of different experimental data sources evidence is often found of significant bias or calibration errors. The uncertainty analysis approach presented in this work is based on the analysis of cumulative probability distributions for output variables of the models involved taking into account the effect of both types of errors. The probability distributions are obtained by performing Monte Carlo simulation coupled with appropriate definitions for the random and systematic errors. The main objectives are to detect the error source with stochastic dominance on the uncertainty propagation and the combined effect on output variables of the models. The results from the case studies analyzed show that the approach is able to distinguish which error type has a more significant effect on the performance of the model. Also, it was found that systematic or calibration errors, if present, cannot be neglected in uncertainty analysis of models dependent on experimental measurements such as chemical and physical properties. The approach can be used to facilitate decision making in fields related to safety factors selection, modeling, experimental data measurement, and experimental design.

Suggested Citation

  • Victor R. Vasquez & Wallace B. Whiting, 2005. "Accounting for Both Random Errors and Systematic Errors in Uncertainty Propagation Analysis of Computer Models Involving Experimental Measurements with Monte Carlo Methods," Risk Analysis, John Wiley & Sons, vol. 25(6), pages 1669-1681, December.
  • Handle: RePEc:wly:riskan:v:25:y:2005:i:6:p:1669-1681
    DOI: 10.1111/j.1539-6924.2005.00704.x
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/j.1539-6924.2005.00704.x
    Download Restriction: no

    File URL: https://libkey.io/10.1111/j.1539-6924.2005.00704.x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. J. C. Helton & D. R. Anderson & M. G. Marietta & R. P. Rechard, 1997. "Performance Assessment for the Waste Isolation Pilot Plant: From Regulation to Calculation for 40 CFR 191.13," Operations Research, INFORMS, vol. 45(2), pages 157-177, April.
    2. F. Owen Hoffman & Jana S. Hammonds, 1994. "Propagation of Uncertainty in Risk Assessments: The Need to Distinguish Between Uncertainty Due to Lack of Knowledge and Uncertainty Due to Variability," Risk Analysis, John Wiley & Sons, vol. 14(5), pages 707-712, October.
    3. William D. Rowe, 1994. "Understanding Uncertainty," Risk Analysis, John Wiley & Sons, vol. 14(5), pages 743-750, October.
    4. Ronald L. Iman & Jon C. Helton, 1988. "An Investigation of Uncertainty and Sensitivity Analysis Techniques for Computer Models," Risk Analysis, John Wiley & Sons, vol. 8(1), pages 71-90, March.
    5. Jon C. Helton, 1994. "Treatment of Uncertainty in Performance Assessments for Complex Systems," Risk Analysis, John Wiley & Sons, vol. 14(4), pages 483-511, August.
    6. Kimberly M. Thompson & David E. Burmaster & Edmund A.C. Crouch3, 1992. "Monte Carlo Techniques for Quantitative Uncertainty Analysis in Public Health Risk Assessments," Risk Analysis, John Wiley & Sons, vol. 12(1), pages 53-63, March.
    7. Alexander I. Shlyakhter, 1994. "An Improved Framework for Uncertainty Analysis: Accounting for Unsuspected Errors," Risk Analysis, John Wiley & Sons, vol. 14(4), pages 441-447, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Per Sander & Bo Bergbäck & Tomas Öberg, 2006. "Uncertain Numbers and Uncertainty in the Selection of Input Distributions—Consequences for a Probabilistic Risk Assessment of Contaminated Land," Risk Analysis, John Wiley & Sons, vol. 26(5), pages 1363-1375, October.
    2. Patrick Frank, 2015. "Negligence, Non-Science, and Consensus Climatology," Energy & Environment, , vol. 26(3), pages 391-415, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. J. C. Helton & D. R. Anderson & H.‐N. Jow & M. G. Marietta & G. Basabilvazo, 1999. "Performance Assessment in Support of the 1996 Compliance Certification Application for the Waste Isolation Pilot Plant," Risk Analysis, John Wiley & Sons, vol. 19(5), pages 959-986, October.
    2. Jon C. Helton & William L. Oberkampf & Jay D. Johnson, 2005. "Competing Failure Risk Analysis Using Evidence Theory," Risk Analysis, John Wiley & Sons, vol. 25(4), pages 973-995, August.
    3. S. Cucurachi & E. Borgonovo & R. Heijungs, 2016. "A Protocol for the Global Sensitivity Analysis of Impact Assessment Models in Life Cycle Assessment," Risk Analysis, John Wiley & Sons, vol. 36(2), pages 357-377, February.
    4. J. C. Helton & F. J. Davis, 2002. "Illustration of Sampling‐Based Methods for Uncertainty and Sensitivity Analysis," Risk Analysis, John Wiley & Sons, vol. 22(3), pages 591-622, June.
    5. Helton, Jon C. & Brooks, Dusty M. & Sallaberry, Cédric J., 2020. "Property values associated with the failure of individual links in a system with multiple weak and strong links," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    6. Hou, Tianfeng & Nuyens, Dirk & Roels, Staf & Janssen, Hans, 2019. "Quasi-Monte Carlo based uncertainty analysis: Sampling efficiency and error estimation in engineering applications," Reliability Engineering and System Safety, Elsevier, vol. 191(C).
    7. Emanuele Borgonovo & Gordon B. Hazen & Elmar Plischke, 2016. "A Common Rationale for Global Sensitivity Measures and Their Estimation," Risk Analysis, John Wiley & Sons, vol. 36(10), pages 1871-1895, October.
    8. Alan H. Stern, 1993. "Re‐evaluation of the Reference Dose for Methylmercury and Assessment of Current Exposure Levels," Risk Analysis, John Wiley & Sons, vol. 13(3), pages 355-364, June.
    9. Julia J. Pet‐Armacost & Jose Sepulveda & Milton Sakude, 1999. "Monte Carlo Sensitivity Analysis of Unknown Parameters in Hazardous Materials Transportation Risk Assessment," Risk Analysis, John Wiley & Sons, vol. 19(6), pages 1173-1184, December.
    10. Wout Slob, 1994. "Uncertainty Analysis in Multiplicative Models," Risk Analysis, John Wiley & Sons, vol. 14(4), pages 571-576, August.
    11. Bas Groot Koerkamp & Theo Stijnen & Milton C. Weinstein & M. G. Myriam Hunink, 2011. "The Combined Analysis of Uncertainty and Patient Heterogeneity in Medical Decision Models," Medical Decision Making, , vol. 31(4), pages 650-661, July.
    12. Helton, Jon C. & Brooks, Dusty M. & Sallaberry, Cédric J., 2020. "Margins associated with loss of assured safety for systems with multiple weak links and strong links," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    13. Sarazin, Gabriel & Morio, Jérôme & Lagnoux, Agnès & Balesdent, Mathieu & Brevault, Loïc, 2021. "Reliability-oriented sensitivity analysis in presence of data-driven epistemic uncertainty," Reliability Engineering and System Safety, Elsevier, vol. 215(C).
    14. S. N. Rai & D. Krewski, 1998. "Uncertainty and Variability Analysis in Multiplicative Risk Models," Risk Analysis, John Wiley & Sons, vol. 18(1), pages 37-45, February.
    15. Emanuele Borgonovo, 2008. "Epistemic Uncertainty in the Ranking and Categorization of Probabilistic Safety Assessment Model Elements: Issues and Findings," Risk Analysis, John Wiley & Sons, vol. 28(4), pages 983-1001, August.
    16. Amirhossein Mokhtari & H. Christopher Frey, 2005. "Sensitivity Analysis of a Two‐Dimensional Probabilistic Risk Assessment Model Using Analysis of Variance," Risk Analysis, John Wiley & Sons, vol. 25(6), pages 1511-1529, December.
    17. Paul S. Price & Cynthia L. Curry & Philip E. Goodrum & Michael N. Gray & Jane I. McCrodden & Natalie W. Harrington & Heather Carlson‐Lynch & Russell E. Keenan, 1996. "Monte Carlo Modeling of Time‐Dependent Exposures Using a Microexposure Event Approach," Risk Analysis, John Wiley & Sons, vol. 16(3), pages 339-348, June.
    18. A. E. Ades & G. Lu & J. P. T. Higgins, 2005. "The Interpretation of Random-Effects Meta-Analysis in Decision Models," Medical Decision Making, , vol. 25(6), pages 646-654, November.
    19. Kimberly M. Thompson, 2002. "Variability and Uncertainty Meet Risk Management and Risk Communication," Risk Analysis, John Wiley & Sons, vol. 22(3), pages 647-654, June.
    20. Andrew E. Smith & P. Barry Ryan & John S. Evans, 1992. "The Effect of Neglecting Correlations When Propagating Uncertainty and Estimating the Population Distribution of Risk," Risk Analysis, John Wiley & Sons, vol. 12(4), pages 467-474, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:riskan:v:25:y:2005:i:6:p:1669-1681. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1111/(ISSN)1539-6924 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.