IDEAS home Printed from https://ideas.repec.org/a/inm/ormoor/v46y2021i2p628-641.html
   My bibliography  Save this article

Learning Optimal Forecast Aggregation in Partial Evidence Environments

Author

Listed:
  • Yakov Babichenko

    (Faculty of Industrial Engineering and Management, Technion - Israel Institute of Technology, Haifa 3200003, Israel)

  • Dan Garber

    (Faculty of Industrial Engineering and Management, Technion - Israel Institute of Technology, Haifa 3200003, Israel)

Abstract

We consider the forecast aggregation problem in repeated settings where the forecasts are of a binary state of nature. In each period multiple experts provide forecasts about the state. The goal of the aggregator is to aggregate those forecasts into a subjective accurate forecast. We assume that the experts are Bayesian and the aggregator is non-Bayesian and ignorant of the information structure (i.e., the distribution over the signals) under which the experts make their forecasts. The aggregator observes the experts’ forecasts only. At the end of each period, the realized state is observed by the aggregator. We focus on the question of whether the aggregator can learn to optimally aggregate the forecasts of the experts, where the optimal aggregation is the Bayesian aggregation that takes into account all the information in the system. We consider the class of partial evidence information structures, where each expert is exposed to a different subset of conditionally independent signals. Our main results are positive: we show that optimal aggregation can be learned in polynomial time in quite a wide range of instances in partial evidence environments. We provide an exact characterization of the instances where optimal learning is possible as well as those where it is impossible.

Suggested Citation

  • Yakov Babichenko & Dan Garber, 2021. "Learning Optimal Forecast Aggregation in Partial Evidence Environments," Mathematics of Operations Research, INFORMS, vol. 46(2), pages 628-641, May.
    • Handle: RePEc:inm:ormoor:v:46:y:2021:i:2:p:628-641
    DOI: 10.1287/moor.2020.1080
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/moor.2020.1080
    Download Restriction: no
    File URL: https://libkey.io/10.1287/moor.2020.1080?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Morris H. DeGroot & Julia Mortera, 1991. "Optimal Linear Opinion Pools," Management Science, INFORMS, vol. 37(5), pages 546-558, May.
    2. Ville A. Satopää & Robin Pemantle & Lyle H. Ungar, 2016. "Modeling Probability Forecasts via Information Diversity," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(516), pages 1623-1633, October.
    3. Robert J. Aumann, 1995. "Repeated Games with Incomplete Information," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262011476, December.
    4. Roopesh Ranjan & Tilmann Gneiting, 2010. "Combining probability forecasts," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 72(1), pages 71-91, January.
    5. Itai Arieli & Yakov Babichenko & Rann Smorodinsky, 2018. "Robust forecast aggregation," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 115(52), pages 12135-12143, December.
    6. Robert F. Bordley, 1982. "A Multiplicative Formula for Aggregating Probability Assessments," Management Science, INFORMS, vol. 28(10), pages 1137-1148, October.
    7. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
    8. Gabriel Carroll, 2015. "Robustness and Linear Contracts," American Economic Review, American Economic Association, vol. 105(2), pages 536-563, February.
    9. Gabriel Carroll, 2017. "Robustness and Separation in Multidimensional Screening," Econometrica, Econometric Society, vol. 85, pages 453-488, March.
    10. Itai Areili & Yakov Babichenko & Rann Smorodinsky, 2017. "Robust Forecast Aggregation," Papers 1710.02838, arXiv.org, revised Feb 2018.
    11. Ernst, Philip & Pemantle, Robin & Satopää, Ville & Ungar, Lyle, 2016. "Bayesian aggregation of two forecasts in the partial information framework," Statistics & Probability Letters, Elsevier, vol. 119(C), pages 170-180.
    12. Satopää, Ville A. & Baron, Jonathan & Foster, Dean P. & Mellers, Barbara A. & Tetlock, Philip E. & Ungar, Lyle H., 2014. "Combining multiple probability predictions using a simple logit model," International Journal of Forecasting, Elsevier, vol. 30(2), pages 344-356.
    13. Arieli, Itai & Babichenko, Yakov & Smorodinsky, Rann, 2020. "Identifiable information structures," Games and Economic Behavior, Elsevier, vol. 120(C), pages 16-27.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Tao Lin & Yiling Chen, 2022. "Sample Complexity of Forecast Aggregation," Papers 2207.13126, arXiv.org, revised Oct 2023.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Satopää, Ville A., 2021. "Improving the wisdom of crowds with analysis of variance of predictions of related outcomes," International Journal of Forecasting, Elsevier, vol. 37(4), pages 1728-1747.
    2. Babichenko, Yakov & Talgam-Cohen, Inbal & Xu, Haifeng & Zabarnyi, Konstantin, 2022. "Regret-minimizing Bayesian persuasion," Games and Economic Behavior, Elsevier, vol. 136(C), pages 226-248.
    3. Ville A. Satopää & Robin Pemantle & Lyle H. Ungar, 2016. "Modeling Probability Forecasts via Information Diversity," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(516), pages 1623-1633, October.
    4. Satopää, Ville A. & Baron, Jonathan & Foster, Dean P. & Mellers, Barbara A. & Tetlock, Philip E. & Ungar, Lyle H., 2014. "Combining multiple probability predictions using a simple logit model," International Journal of Forecasting, Elsevier, vol. 30(2), pages 344-356.
    5. Wang, Xiaoqian & Hyndman, Rob J. & Li, Feng & Kang, Yanfei, 2023. "Forecast combinations: An over 50-year review," International Journal of Forecasting, Elsevier, vol. 39(4), pages 1518-1547.
    6. Henrique de Oliveira & Yuhta Ishii & Xiao Lin, 2021. "Robust Merging of Information," Papers 2106.00088, arXiv.org.
    7. Stephen Hora & Erim Kardeş, 2015. "Calibration, sharpness and the weighting of experts in a linear opinion pool," Annals of Operations Research, Springer, vol. 229(1), pages 429-450, June.
    8. Ville A. Satopää & Marat Salikhov & Philip E. Tetlock & Barbara Mellers, 2021. "Bias, Information, Noise: The BIN Model of Forecasting," Management Science, INFORMS, vol. 67(12), pages 7599-7618, December.
    9. Federico Bassetti & Roberto Casarin & Francesco Ravazzolo, 2019. "Density Forecasting," BEMPS - Bozen Economics & Management Paper Series BEMPS59, Faculty of Economics and Management at the Free University of Bozen.
    10. Carroll, Gabriel, 2019. "Robust incentives for information acquisition," Journal of Economic Theory, Elsevier, vol. 181(C), pages 382-420.
    11. Itai Areili & Yakov Babichenko & Rann Smorodinsky, 2017. "Robust Forecast Aggregation," Papers 1710.02838, arXiv.org, revised Feb 2018.
    12. Jared A. Beekman & Ronald F. A. Woodaman & Dennis M. Buede, 2020. "A Review of Probabilistic Opinion Pooling Algorithms with Application to Insider Threat Detection," Decision Analysis, INFORMS, vol. 17(1), pages 39-55, March.
    13. Arieli, Itai & Babichenko, Yakov & Smorodinsky, Rann, 2020. "Identifiable information structures," Games and Economic Behavior, Elsevier, vol. 120(C), pages 16-27.
    14. Lahiri, Kajal & Yang, Liu, 2013. "Forecasting Binary Outcomes," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1025-1106, Elsevier.
    15. Levy, Gilat & Moreno de Barreda, Inés & Razin, Ronny, 2022. "Persuasion with correlation neglect: a full manipulation result," LSE Research Online Documents on Economics 111551, London School of Economics and Political Science, LSE Library.
    16. Gael M. Martin & David T. Frazier & Ruben Loaiza-Maya & Florian Huber & Gary Koop & John Maheu & Didier Nibbering & Anastasios Panagiotelis, 2023. "Bayesian Forecasting in the 21st Century: A Modern Review," Monash Econometrics and Business Statistics Working Papers 1/23, Monash University, Department of Econometrics and Business Statistics.
    17. Donnacha Bolger & Brett Houlding, 2016. "Reliability updating in linear opinion pooling for multiple decision makers," Journal of Risk and Reliability, , vol. 230(3), pages 309-322, June.
    18. Tang, Rui & Zhang, Mu, 2021. "Maxmin implementation," Journal of Economic Theory, Elsevier, vol. 194(C).
    19. Yang, Dazhi & van der Meer, Dennis, 2021. "Post-processing in solar forecasting: Ten overarching thinking tools," Renewable and Sustainable Energy Reviews, Elsevier, vol. 140(C).
    20. Garratt, Anthony & Henckel, Timo & Vahey, Shaun P., 2023. "Empirically-transformed linear opinion pools," International Journal of Forecasting, Elsevier, vol. 39(2), pages 736-753.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormoor:v:46:y:2021:i:2:p:628-641. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.