IDEAS home Printed from https://ideas.repec.org/a/wly/fufsci/v5y2023i1ne139.html
   My bibliography  Save this article

Averaging quantiles, variance shrinkage, and overconfidence

Author

Listed:
  • Roger M. Cooke

Abstract

Averaging quantiles as a way of combining experts' judgments is studied both mathematically and empirically. Quantile averaging is equivalent to taking the harmonic mean of densities evaluated at quantile points. A variance shrinkage law is established between equal and harmonic weighting. Data from 49 post‐2006 studies are extended to include harmonic weighting in addition to equal and performance‐based weighting. It emerges that harmonic weighting has the highest average information and degraded statistical accuracy. The hypothesis that the quantile average is statistically accurate would be rejected at the 5% level in 28 studies and at the 0.1% level in 15 studies. For performance weighting, these numbers are 3 and 1, for equal weighting 2 and 1.

Suggested Citation

  • Roger M. Cooke, 2023. "Averaging quantiles, variance shrinkage, and overconfidence," Futures & Foresight Science, John Wiley & Sons, vol. 5(1), March.
    • Handle: RePEc:wly:fufsci:v:5:y:2023:i:1:n:e139
    DOI: 10.1002/ffo2.139
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/ffo2.139
    Download Restriction: no
    File URL: https://libkey.io/10.1002/ffo2.139?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Colson, Abigail R. & Cooke, Roger M., 2017. "Cross validation for the classical model of structured expert judgment," Reliability Engineering and System Safety, Elsevier, vol. 163(C), pages 109-120.
    2. De Gooijer, Jan G. & Zerom, Dawit, 2019. "Semiparametric quantile averaging in the presence of high-dimensional predictors," International Journal of Forecasting, Elsevier, vol. 35(3), pages 891-909.
    3. Cooke, Roger M. & Marti, Deniz & Mazzuchi, Thomas, 2021. "Expert forecasting with and without uncertainty quantification and weighting: What do the data say?," International Journal of Forecasting, Elsevier, vol. 37(1), pages 378-387.
    4. Kenneth C. Lichtendahl & Yael Grushka-Cockayne & Robert L. Winkler, 2013. "Is It Better to Average Probabilities or Quantiles?," Management Science, INFORMS, vol. 59(7), pages 1594-1611, July.
    5. Michael Oppenheimer & Christopher M. Little & Roger M. Cooke, 2016. "Expert judgement and uncertainty quantification for climate change," Nature Climate Change, Nature, vol. 6(5), pages 445-451, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ren, Xin & Nane, Gabriela F. & Terwel, Karel C. & van Gelder, Pieter H.A.J.M., 2024. "Measuring the impacts of human and organizational factors on human errors in the Dutch construction industry using structured expert judgement," Reliability Engineering and System Safety, Elsevier, vol. 244(C).
    2. Fathy, Mohammad & Kazemzadeh Haghighi, Foojan & Ahmadi, Mohammad, 2024. "Uncertainty quantification of reservoir performance using machine learning algorithms and structured expert judgment," Energy, Elsevier, vol. 288(C).
    3. Hathout, Michel & Vuillet, Marc & Carvajal, Claudio & Peyras, Laurent & Diab, Youssef, 2019. "Expert judgments calibration and combination for assessment of river levee failure probability," Reliability Engineering and System Safety, Elsevier, vol. 188(C), pages 377-392.
    4. Despoina Makariou & Pauline Barrieu & George Tzougas, 2021. "A Finite Mixture Modelling Perspective for Combining Experts’ Opinions with an Application to Quantile-Based Risk Measures," Risks, MDPI, vol. 9(6), pages 1-25, June.
    5. Gabriela F. Nane & Roger M. Cooke, 2024. "Scoring rules and performance, new analysis of expert judgment data," Futures & Foresight Science, John Wiley & Sons, vol. 6(4), December.
    6. Timothy McDaniels, 2021. "Four Decades of Transformation in Decision Analytic Practice for Societal Risk Management," Risk Analysis, John Wiley & Sons, vol. 41(3), pages 491-502, March.
    7. Cooke, Roger M. & Marti, Deniz & Mazzuchi, Thomas, 2021. "Expert forecasting with and without uncertainty quantification and weighting: What do the data say?," International Journal of Forecasting, Elsevier, vol. 37(1), pages 378-387.
    8. Baker, Erin & Bosetti, Valentina & Salo, Ahti, 2016. "Finding Common Ground when Experts Disagree: Belief Dominance over Portfolios of Alternatives," MITP: Mitigation, Innovation and Transformation Pathways 243147, Fondazione Eni Enrico Mattei (FEEM).
    9. Pindyck, Robert S., 2019. "The social cost of carbon revisited," Journal of Environmental Economics and Management, Elsevier, vol. 94(C), pages 140-160.
    10. Tony E. Wong & Alexander M. R. Bakker & Klaus Keller, 2017. "Impacts of Antarctic fast dynamics on sea-level projections and coastal flood defense," Climatic Change, Springer, vol. 144(2), pages 347-364, September.
    11. Uniejewski, Bartosz & Weron, Rafał, 2021. "Regularized quantile regression averaging for probabilistic electricity price forecasting," Energy Economics, Elsevier, vol. 95(C).
    12. Funk, Patrick & Davis, Alex & Vaishnav, Parth & Dewitt, Barry & Fuchs, Erica, 2020. "Individual inconsistency and aggregate rationality: Overcoming inconsistencies in expert judgment at the technical frontier," Technological Forecasting and Social Change, Elsevier, vol. 155(C).
    13. Tomasz Serafin & Bartosz Uniejewski & Rafał Weron, 2019. "Averaging Predictive Distributions Across Calibration Windows for Day-Ahead Electricity Price Forecasting," Energies, MDPI, vol. 12(13), pages 1-12, July.
    14. David M Costello & Konrad J Kulacki & Mary E McCarthy & Scott D Tiegs & Bradley J Cardinale, 2018. "Ranking stressor impacts on periphyton structure and function with mesocosm experiments and environmental-change forecasts," PLOS ONE, Public Library of Science, vol. 13(9), pages 1-18, September.
    15. Patrick Afflerbach & Christopher Dun & Henner Gimpel & Dominik Parak & Johannes Seyfried, 2021. "A Simulation-Based Approach to Understanding the Wisdom of Crowds Phenomenon in Aggregating Expert Judgment," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 63(4), pages 329-348, August.
    16. Konrad Bogner & Katharina Liechti & Luzi Bernhard & Samuel Monhart & Massimiliano Zappa, 2018. "Skill of Hydrological Extended Range Forecasts for Water Resources Management in Switzerland," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 32(3), pages 969-984, February.
    17. Wang, Shengjie & Kang, Yanfei & Petropoulos, Fotios, 2024. "Combining probabilistic forecasts of intermittent demand," European Journal of Operational Research, Elsevier, vol. 315(3), pages 1038-1048.
    18. Malte Knüppel & Fabian Krüger, 2022. "Forecast uncertainty, disagreement, and the linear pool," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(1), pages 23-41, January.
    19. Weronika Nitka & Rafał Weron, 2023. "Combining predictive distributions of electricity prices. Does minimizing the CRPS lead to optimal decisions in day-ahead bidding?," Operations Research and Decisions, Wroclaw University of Science and Technology, Faculty of Management, vol. 33(3), pages 105-118.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:fufsci:v:5:y:2023:i:1:n:e139. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1002/(ISSN)2573-5152 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.