IDEAS home Printed from https://ideas.repec.org/a/eee/reensy/v188y2019icp377-392.html
   My bibliography  Save this article

Expert judgments calibration and combination for assessment of river levee failure probability

Author

Listed:
  • Hathout, Michel
  • Vuillet, Marc
  • Carvajal, Claudio
  • Peyras, Laurent
  • Diab, Youssef

Abstract

In civil engineering, the utilisation of expert judgement is important for evaluating the reliability of structures whose failure behaviour is little known and difficult to quantify. This article presents the application of the approaches of elicitation, calibration and aggregation of expert opinions to evaluate the reliability of river levees. These approaches have seldom been used in the field of French hydraulic structures whereas there is growing interest for them in other countries, concerning structures reliability and other fields such as aerospace industry, nuclear industry, hydrology, statistics, economics, psychology etc. This article proposes a quantitative approach to the use of expert judgement for evaluating river levee failure probability. An application to the case of an existing river levee is presented. The approach developed assesses the expert opinions according to their calibration (statistical accuracy) and their informativeness, then aggregates them in a single calibrated opinion. Applying the proposed weighting and aggregation procedure reduces variability in the probability of failure estimate, then it provides a better probabilistic distribution of aggregated expert opinion than of individual expert opinions. The results allow identifying a trend of aggregated expert opinions that point towards over- or lack of confidence.

Suggested Citation

  • Hathout, Michel & Vuillet, Marc & Carvajal, Claudio & Peyras, Laurent & Diab, Youssef, 2019. "Expert judgments calibration and combination for assessment of river levee failure probability," Reliability Engineering and System Safety, Elsevier, vol. 188(C), pages 377-392.
    • Handle: RePEc:eee:reensy:v:188:y:2019:i:c:p:377-392
    DOI: 10.1016/j.ress.2019.03.019
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0951832018303296
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ress.2019.03.019?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Richard P. Larrick & Jack B. Soll, 2006. "Erratum--Intuitions About Combining Opinions: Misappreciation of the Averaging Principle," Management Science, INFORMS, vol. 52(2), pages 309-310, February.
    2. John Quigley & Abigail Colson & Willy Aspinall & Roger M. Cooke, 2018. "Elicitation in the Classical Model," International Series in Operations Research & Management Science, in: Luis C. Dias & Alec Morton & John Quigley (ed.), Elicitation, chapter 0, pages 15-36, Springer.
    3. Roger Cooke & Eric Jager, 1998. "A Probabilistic Model for the Failure Frequency of Underground Gas Pipelines," Risk Analysis, John Wiley & Sons, vol. 18(4), pages 511-527, August.
    4. Lin, Shi-Woei & Bier, Vicki M., 2008. "A study of expert overconfidence," Reliability Engineering and System Safety, Elsevier, vol. 93(5), pages 711-721.
    5. Cooke, Roger M. & Goossens, Louis L.H.J., 2008. "TU Delft expert judgment data base," Reliability Engineering and System Safety, Elsevier, vol. 93(5), pages 657-674.
    6. Garthwaite, Paul H. & Kadane, Joseph B. & O'Hagan, Anthony, 2005. "Statistical Methods for Eliciting Probability Distributions," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 680-701, June.
    7. Heiko Apel & Annegret Thieken & Bruno Merz & Günter Blöschl, 2006. "A Probabilistic Modelling System for Assessing Flood Risks," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 38(1), pages 79-100, May.
    8. Alexina J. Mason & Manuel Gomes & Richard Grieve & James R. Carpenter, 2018. "A Bayesian framework for health economic evaluation in studies with missing data," Health Economics, John Wiley & Sons, Ltd., vol. 27(11), pages 1670-1683, November.
    9. Colson, Abigail R. & Cooke, Roger M., 2017. "Cross validation for the classical model of structured expert judgment," Reliability Engineering and System Safety, Elsevier, vol. 163(C), pages 109-120.
    10. Soll, Jack B., 1996. "Determinants of Overconfidence and Miscalibration: The Roles of Random Error and Ecological Structure," Organizational Behavior and Human Decision Processes, Elsevier, vol. 65(2), pages 117-137, February.
    11. Forys, Monika B. & Kurowicka, Dorota & Peppelman, Bas, 2013. "A probabilistic model for a gas explosion due to leakages in the grey cast iron gas mains," Reliability Engineering and System Safety, Elsevier, vol. 119(C), pages 270-279.
    12. Kenneth C. Lichtendahl & Yael Grushka-Cockayne & Robert L. Winkler, 2013. "Is It Better to Average Probabilities or Quantiles?," Management Science, INFORMS, vol. 59(7), pages 1594-1611, July.
    13. Abigail R Colson & Roger M Cooke, 2018. "Expert Elicitation: Using the Classical Model to Validate Experts’ Judgments," Review of Environmental Economics and Policy, Association of Environmental and Resource Economists, vol. 12(1), pages 113-132.
    14. Richard P. Larrick & Jack B. Soll, 2006. "Intuitions About Combining Opinions: Misappreciation of the Averaging Principle," Management Science, INFORMS, vol. 52(1), pages 111-127, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jari Lyytimäki & Kirsi-Marja Lonkila & Eeva Furman & Kaisa Korhonen-Kurki & Satu Lähteenoja, 2021. "Untangling the interactions of sustainability targets: synergies and trade-offs in the Northern European context," Environment, Development and Sustainability: A Multidisciplinary Approach to the Theory and Practice of Sustainable Development, Springer, vol. 23(3), pages 3458-3473, March.
    2. Rajabzadeh, Vida & Hekmatzadeh, Ali Akbar & Tabatabaie Shourijeh, Piltan & Torabi Haghighi, Ali, 2023. "Introducing a probabilistic framework to measure dam overtopping risk for dams benefiting from dual spillways," Reliability Engineering and System Safety, Elsevier, vol. 231(C).
    3. Rongen, G. & Morales-Nápoles, O. & Kok, M., 2022. "Expert judgment-based reliability analysis of the Dutch flood defense system," Reliability Engineering and System Safety, Elsevier, vol. 224(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jeremy Rohmer & Eric Chojnacki, 2021. "Forecast of environment systems using expert judgements: performance comparison between the possibilistic and the classical model," Environment Systems and Decisions, Springer, vol. 41(1), pages 131-146, March.
    2. Patrick Afflerbach & Christopher Dun & Henner Gimpel & Dominik Parak & Johannes Seyfried, 2021. "A Simulation-Based Approach to Understanding the Wisdom of Crowds Phenomenon in Aggregating Expert Judgment," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 63(4), pages 329-348, August.
    3. Elena Verdolini & Laura Díaz Anadón & Erin Baker & Valentina Bosetti & Lara Aleluia Reis, 2018. "Future Prospects for Energy Technologies: Insights from Expert Elicitations," Review of Environmental Economics and Policy, Association of Environmental and Resource Economists, vol. 12(1), pages 133-153.
    4. Cooke, Roger M. & Marti, Deniz & Mazzuchi, Thomas, 2021. "Expert forecasting with and without uncertainty quantification and weighting: What do the data say?," International Journal of Forecasting, Elsevier, vol. 37(1), pages 378-387.
    5. Wang, Xiaoqian & Hyndman, Rob J. & Li, Feng & Kang, Yanfei, 2023. "Forecast combinations: An over 50-year review," International Journal of Forecasting, Elsevier, vol. 39(4), pages 1518-1547.
    6. Yael Grushka-Cockayne & Kenneth C. Lichtendahl Jr. & Victor Richmond R. Jose & Robert L. Winkler, 2017. "Quantile Evaluation, Sensitivity to Bracketing, and Sharing Business Payoffs," Operations Research, INFORMS, vol. 65(3), pages 712-728, June.
    7. Misuri, Alessio & Landucci, Gabriele & Cozzani, Valerio, 2020. "Assessment of safety barrier performance in Natech scenarios," Reliability Engineering and System Safety, Elsevier, vol. 193(C).
    8. Eggstaff, Justin W. & Mazzuchi, Thomas A. & Sarkani, Shahram, 2014. "The effect of the number of seed variables on the performance of Cooke′s classical model," Reliability Engineering and System Safety, Elsevier, vol. 121(C), pages 72-82.
    9. Peter Harrison Howard & Derek Sylvan, 2020. "Wisdom of the experts: Using survey responses to address positive and normative uncertainties in climate-economic models," Climatic Change, Springer, vol. 162(2), pages 213-232, September.
    10. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    11. Despoina Makariou & Pauline Barrieu & George Tzougas, 2021. "A Finite Mixture Modelling Perspective for Combining Experts’ Opinions with an Application to Quantile-Based Risk Measures," Risks, MDPI, vol. 9(6), pages 1-25, June.
    12. Taylor, James W. & Taylor, Kathryn S., 2023. "Combining probabilistic forecasts of COVID-19 mortality in the United States," European Journal of Operational Research, Elsevier, vol. 304(1), pages 25-41.
    13. Mohammad Yazdi, 2019. "A review paper to examine the validity of Bayesian network to build rational consensus in subjective probabilistic failure analysis," International Journal of System Assurance Engineering and Management, Springer;The Society for Reliability, Engineering Quality and Operations Management (SREQOM),India, and Division of Operation and Maintenance, Lulea University of Technology, Sweden, vol. 10(1), pages 1-18, February.
    14. Christopher W. Karvetski & Kenneth C. Olson & David R. Mandel & Charles R. Twardy, 2013. "Probabilistic Coherence Weighting for Optimizing Expert Forecasts," Decision Analysis, INFORMS, vol. 10(4), pages 305-326, December.
    15. Jordan Tong & Daniel Feiler, 2017. "A Behavioral Model of Forecasting: Naive Statistics on Mental Samples," Management Science, INFORMS, vol. 63(11), pages 3609-3627, November.
    16. Claire Copeland & Britta Turner & Gareth Powells & Kevin Wilson, 2022. "In Search of Complementarity: Insights from an Exercise in Quantifying Qualitative Energy Futures," Energies, MDPI, vol. 15(15), pages 1-21, July.
    17. Armstrong, J. Scott & Green, Kesten C. & Graefe, Andreas, 2015. "Golden rule of forecasting: Be conservative," Journal of Business Research, Elsevier, vol. 68(8), pages 1717-1731.
    18. Atanasov, Pavel & Witkowski, Jens & Ungar, Lyle & Mellers, Barbara & Tetlock, Philip, 2020. "Small steps to accuracy: Incremental belief updaters are better forecasters," Organizational Behavior and Human Decision Processes, Elsevier, vol. 160(C), pages 19-35.
    19. repec:cup:judgdm:v:14:y:2019:i:4:p:395-411 is not listed on IDEAS
    20. Gino, Francesca, 2008. "Do we listen to advice just because we paid for it? The impact of advice cost on its use," Organizational Behavior and Human Decision Processes, Elsevier, vol. 107(2), pages 234-245, November.
    21. repec:cup:judgdm:v:8:y:2013:i:2:p:91-105 is not listed on IDEAS
    22. Phanish Puranam, 2021. "Human–AI collaborative decision-making as an organization design problem," Journal of Organization Design, Springer;Organizational Design Community, vol. 10(2), pages 75-80, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:reensy:v:188:y:2019:i:c:p:377-392. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/reliability-engineering-and-system-safety .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.