IDEAS home Printed from https://ideas.repec.org/a/wly/riskan/v19y1999i1p109-130.html
   My bibliography  Save this article

Methods for Characterizing Variability and Uncertainty: Comparison of Bootstrap Simulation and Likelihood‐Based Approaches

Author

Listed:
  • H. Christopher Frey
  • David E. Burmaster

Abstract

Variability arises due to differences in the value of a quantity among different members of a population. Uncertainty arises due to lack of knowledge regarding the true value of a quantity for a given member of a population. We describe and evaluate two methods for quantifying both variability and uncertainty. These methods, bootstrapsimulation and a likelihood‐based method, are applied to three datasets. The datasetsinclude a synthetic sample of 19 values from a Lognormal distribution, a sample of nine values obtained from measurements of the PCB concentration in leafy produce, and asample of five values for the partitioning of chromium in the flue gas desulfurization system of coal‐fired power plants. For each of these datasets, we employ the two methods to characterize uncertainty in the arithmetic mean and standard deviation, cumulative distribution functions based upon fitted parametric distributions, the 95th percentile of variability, and the 63rd percentile of uncertainty for the 81st percentile of variability. The latter is intended to show that it is possible to describe anypoint within the uncertain frequency distribution by specifying an uncertainty percentile and a Variability percentile. Using the bootstrap method, we compare results based upon use of the method of matching moments and the method of maximum likelihood for fitting distributions to data. Our results indicate that with only 5‐19 data pointsas in the datasets we have evaluated, there is substantial uncertainty based upon random sampling error. Both the boostrap and likelihood‐based approaches yield comparable uncertainty estimates in most cases.

Suggested Citation

  • H. Christopher Frey & David E. Burmaster, 1999. "Methods for Characterizing Variability and Uncertainty: Comparison of Bootstrap Simulation and Likelihood‐Based Approaches," Risk Analysis, John Wiley & Sons, vol. 19(1), pages 109-130, February.
    • Handle: RePEc:wly:riskan:v:19:y:1999:i:1:p:109-130
    DOI: 10.1111/j.1539-6924.1999.tb00393.x
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/j.1539-6924.1999.tb00393.x
    Download Restriction: no
    File URL: https://libkey.io/10.1111/j.1539-6924.1999.tb00393.x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. F. Owen Hoffman & Jana S. Hammonds, 1994. "Propagation of Uncertainty in Risk Assessments: The Need to Distinguish Between Uncertainty Due to Lack of Knowledge and Uncertainty Due to Variability," Risk Analysis, John Wiley & Sons, vol. 14(5), pages 707-712, October.
    2. David L. Macintosh & Glenn W. Suter & F. Owen Hoffman, 1994. "Uses of Probabilistic Exposure Models in Ecological Risk Assessments of Contaminated Sites," Risk Analysis, John Wiley & Sons, vol. 14(4), pages 405-419, August.
    3. Kenneth T. Bogen & Robert C. Spear, 1987. "Integrating Uncertainty and Interindividual Variability in Environmental Risk Assessment," Risk Analysis, John Wiley & Sons, vol. 7(4), pages 427-436, December.
    4. Kenneth T. Bogen, 1995. "Methods to Approximate Joint Uncertainty and Variability in Risk," Risk Analysis, John Wiley & Sons, vol. 15(3), pages 411-419, June.
    5. Thomas E. McKone, 1994. "Uncertainty and Variability in Human Exposures to Soil Contaminants Through Home‐Grown Food: A Monte Carlo Assessment," Risk Analysis, John Wiley & Sons, vol. 14(4), pages 449-463, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Clémence Sophie Rigaux Ancelet & Frédéric Carlin & Christophe Nguyen‐thé & Isabelle Albert, 2013. "Inferring an Augmented Bayesian Network to Confront a Complex Quantitative Microbial Risk Assessment Model with Durability Studies: Application to Bacillus Cereus on a Courgette Purée Production Chain," Risk Analysis, John Wiley & Sons, vol. 33(5), pages 877-892, May.
    2. Nicolas Miconnet & Marie Cornu & Annie Beaufort & Laurent Rosso & Jean‐Baptiste Denis, 2005. "Uncertainty Distribution Associated with Estimating a Proportion in Microbial Risk Assessment," Risk Analysis, John Wiley & Sons, vol. 25(1), pages 39-48, February.
    3. Régis Pouillot & Nicolas Miconnet & Anne‐Laure Afchain & Marie Laure Delignette‐Muller & Annie Beaufort & Laurent Rosso & Jean‐Baptiste Denis & Marie Cornu, 2007. "Quantitative Risk Assessment of Listeria monocytogenes in French Cold‐Smoked Salmon: I. Quantitative Exposure Assessment," Risk Analysis, John Wiley & Sons, vol. 27(3), pages 683-700, June.
    4. Xavier Romão & Esmeralda Paupério, 2016. "A framework to assess quality and uncertainty in disaster loss data," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 83(2), pages 1077-1102, September.
    5. Nicola Pedroni & Enrico Zio & Alberto Pasanisi & Mathieu Couplet, 2017. "A critical discussion and practical recommendations on some issues relevant to the non-probabilistic treatment of uncertainty in engineering risk assessment," Post-Print hal-01652230, HAL.
    6. Modarres, Reza & Nayak, Tapan K. & Gastwirth, Joseph L., 2002. "Estimation of upper quantiles under model and parameter uncertainty," Computational Statistics & Data Analysis, Elsevier, vol. 39(4), pages 529-554, June.
    7. McKenna, Claire & Chalabi, Zaid & Epstein, David & Claxton, Karl, 2010. "Budgetary policies and available actions: A generalisation of decision rules for allocation and research decisions," Journal of Health Economics, Elsevier, vol. 29(1), pages 170-181, January.
    8. Jason R. W. Merrick & J. Rene Van Dorp & Varun Dinesh, 2005. "Assessing Uncertainty in Simulation‐Based Maritime Risk Assessment," Risk Analysis, John Wiley & Sons, vol. 25(3), pages 731-743, June.
    9. Michael Greenberg & Karen Lowrie, 2011. "Celebrating Three Decades of Public Policy‐Oriented Interdisciplinary Research," Risk Analysis, John Wiley & Sons, vol. 31(1), pages 7-11, January.
    10. Jason R. W. Merrick & Rene Van Dorp, 2006. "Speaking the Truth in Maritime Risk Assessment," Risk Analysis, John Wiley & Sons, vol. 26(1), pages 223-237, February.
    11. Chalabi, Zaid & Epstein, David & McKenna, Claire & Claxton, Karl, 2008. "Uncertainty and value of information when allocating resources within and between healthcare programmes," European Journal of Operational Research, Elsevier, vol. 191(2), pages 530-539, December.
    12. Arwa S. Sayegh & Richard D. Connors & James E. Tate, 2018. "Uncertainty Propagation from the Cell Transmission Traffic Flow Model to Emission Predictions: A Data-Driven Approach," Service Science, INFORMS, vol. 52(6), pages 1327-1346, December.
    13. Nicola Pedroni & Enrico Zio & Alberto Pasanisi & Mathieu Couplet, 2017. "A Critical Discussion and Practical Recommendations on Some Issues Relevant to the Nonprobabilistic Treatment of Uncertainty in Engineering Risk Assessment," Risk Analysis, John Wiley & Sons, vol. 37(7), pages 1315-1340, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Paul S. Price & Steave H. Su & Jeff R. Harrington & Russell E. Keenan, 1996. "Uncertainty and Variation in Indirect Exposure Assessments: An Analysis of Exposure to Tetrachlorodibenzo‐p‐Dioxin from a Beef Consumption Pathway," Risk Analysis, John Wiley & Sons, vol. 16(2), pages 263-277, April.
    2. J. C. Helton & F. J. Davis, 2002. "Illustration of Sampling‐Based Methods for Uncertainty and Sensitivity Analysis," Risk Analysis, John Wiley & Sons, vol. 22(3), pages 591-622, June.
    3. Junyu Zheng & H. Christopher Frey, 2005. "Quantitative Analysis of Variability and Uncertainty with Known Measurement Error: Methodology and Case Study," Risk Analysis, John Wiley & Sons, vol. 25(3), pages 663-675, June.
    4. Bas Groot Koerkamp & Theo Stijnen & Milton C. Weinstein & M. G. Myriam Hunink, 2011. "The Combined Analysis of Uncertainty and Patient Heterogeneity in Medical Decision Models," Medical Decision Making, , vol. 31(4), pages 650-661, July.
    5. Junyu Zheng & H. Christopher Frey, 2004. "Quantification of Variability and Uncertainty Using Mixture Distributions: Evaluation of Sample Size, Mixing Weights, and Separation Between Components," Risk Analysis, John Wiley & Sons, vol. 24(3), pages 553-571, June.
    6. Kenneth T. Bogen, 2005. "Risk Analysis for Environmental Health Triage," Risk Analysis, John Wiley & Sons, vol. 25(5), pages 1085-1095, October.
    7. S. N. Rai & D. Krewski, 1998. "Uncertainty and Variability Analysis in Multiplicative Risk Models," Risk Analysis, John Wiley & Sons, vol. 18(1), pages 37-45, February.
    8. Paul S. Price & Cynthia L. Curry & Philip E. Goodrum & Michael N. Gray & Jane I. McCrodden & Natalie W. Harrington & Heather Carlson‐Lynch & Russell E. Keenan, 1996. "Monte Carlo Modeling of Time‐Dependent Exposures Using a Microexposure Event Approach," Risk Analysis, John Wiley & Sons, vol. 16(3), pages 339-348, June.
    9. A. E. Ades & G. Lu & J. P. T. Higgins, 2005. "The Interpretation of Random-Effects Meta-Analysis in Decision Models," Medical Decision Making, , vol. 25(6), pages 646-654, November.
    10. Lee, Chang-Ju & Lee, Kun Jai, 2006. "Application of Bayesian network to the probabilistic risk assessment of nuclear waste disposal," Reliability Engineering and System Safety, Elsevier, vol. 91(5), pages 515-532.
    11. Zhou, Yuekuan & Zheng, Siqian, 2020. "Uncertainty study on thermal and energy performances of a deterministic parameters based optimal aerogel glazing system using machine-learning method," Energy, Elsevier, vol. 193(C).
    12. T. Chatzivasileiadis & F. Estrada & M. W. Hofkes & R. S. J. Tol, 2019. "Systematic Sensitivity Analysis of the Full Economic Impacts of Sea Level Rise," Computational Economics, Springer;Society for Computational Economics, vol. 53(3), pages 1183-1217, March.
    13. Jingwen Song & Zhenzhou Lu & Pengfei Wei & Yanping Wang, 2015. "Global sensitivity analysis for model with random inputs characterized by probability-box," Journal of Risk and Reliability, , vol. 229(3), pages 237-253, June.
    14. Brent Finley & Deborah Proctor & Paul Scott & Natalie Harrington & Dennis Paustenbach & Paul Price, 1994. "Recommended Distributions for Exposure Factors Frequently Used in Health Risk Assessment," Risk Analysis, John Wiley & Sons, vol. 14(4), pages 533-553, August.
    15. Helton, Jon C. & Brooks, Dusty M. & Sallaberry, Cédric J., 2020. "Property values associated with the failure of individual links in a system with multiple weak and strong links," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    16. Maged M. Hamed & Philip B. Bedient, 1997. "On the Effect of Probability Distributions of Input Variables in Public Health Risk Assessment," Risk Analysis, John Wiley & Sons, vol. 17(1), pages 97-105, February.
    17. Kenneth T. Bogen, 2014. "Does EPA Underestimate Cancer Risks by Ignoring Susceptibility Differences?," Risk Analysis, John Wiley & Sons, vol. 34(10), pages 1780-1784, October.
    18. T. E. McKone & J. I. Daniels & M. Goldman, 1996. "Uncertainties in the Link Between Global Climate Change and Predicted Health Risks from Pollution: Hexachlorobenzene (HCB) Case Study Using a Fugacity Model," Risk Analysis, John Wiley & Sons, vol. 16(3), pages 377-393, June.
    19. Frédéric Dor & Pascal Empereur‐Bissonnet & Denis Zmirou & Vincent Nedellec & Jean‐Marie Haguenoer & Frans Jongeneelen & Alain Person & William Dab & Colin Ferguson, 2003. "Validation of Multimedia Models Assessing Exposure to PAHs—The SOLEX Study," Risk Analysis, John Wiley & Sons, vol. 23(5), pages 1047-1057, October.
    20. Lisa M. Funk & Richard Sedman & Jill A. J. Beals & Robert Fountain, 1998. "Quantifying the Distribution of Inhalation Exposure in Human Populations: 2. Distributions of Time Spent by Adults, Adolescents, and Children at Home, at Work, and at School," Risk Analysis, John Wiley & Sons, vol. 18(1), pages 47-56, February.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:riskan:v:19:y:1999:i:1:p:109-130. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1111/(ISSN)1539-6924 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.