IDEAS home Printed from https://ideas.repec.org/a/inm/ordeca/v3y2006i3p145-162.html
   My bibliography  Save this article

To Bayes or Not to Bayes? A Comparison of Two Classes of Models of Information Aggregation

Author

Listed:
  • David V. Budescu

    (Department of Psychology, University of Illinois at Urbana-Champaign, 603 E. Daniel Street, Champaign, Illinois 61820)

  • Hsiu-Ting Yu

    (Department of Psychology, University of Illinois at Urbana-Champaign, 603 E. Daniel Street, Champaign, Illinois 61820)

Abstract

We model the aggregation process used by individual decision makers (DMs) who obtain probabilistic information from multiple, possibly nonindependent, sources. We distinguish between two qualitatively different aggregation approaches: compromising , by averaging the advisors’ opinions, and combining the forecasts according to a naïve implementation of Bayes rule that assumes conditional independence between advisors. The DMs in our studies received forecasts from two or three advisors who had access to multiple diagnostic cues, and made a large number of decisions. Our data are unusually rich in many respects since the studies involve natural sampling of cues with various levels of dependence and various patterns of information overlap. This provides an excellent opportunity to compare the quality of these models. Overall, the DMs’ judgments were closest to the averaging model but, clearly, they did not rely exclusively on this model. The DMs’ aggregates were more in line with the naïve Bayes rule when the advisors provided extreme forecasts were highly consistent with each other, and induced high levels of confidence. On the other hand, when the advisors disagreed with each other, the DMs were less confident and their aggregates were predicted well by a simple averaging rule.

Suggested Citation

  • David V. Budescu & Hsiu-Ting Yu, 2006. "To Bayes or Not to Bayes? A Comparison of Two Classes of Models of Information Aggregation," Decision Analysis, INFORMS, vol. 3(3), pages 145-162, September.
  • Handle: RePEc:inm:ordeca:v:3:y:2006:i:3:p:145-162
    DOI: 10.1287/deca.1060.0074
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/deca.1060.0074
    Download Restriction: no

    File URL: https://libkey.io/10.1287/deca.1060.0074?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Sniezek, Janet A. & Buckley, Timothy, 1995. "Cueing and Cognitive Conflict in Judge-Advisor Decision Making," Organizational Behavior and Human Decision Processes, Elsevier, vol. 62(2), pages 159-174, May.
    2. Peter A. Morris, 1983. "An Axiomatic Approach to Expert Resolution," Management Science, INFORMS, vol. 29(1), pages 24-32, January.
    3. Budescu, David V. & Rantilla, Adrian K. & Yu, Hsiu-Ting & Karelitz, Tzur M., 2003. "The effects of asymmetry among advisors on the aggregation of their opinions," Organizational Behavior and Human Decision Processes, Elsevier, vol. 90(1), pages 178-194, January.
    4. Dennis P. Slevin & Larry W. Boone & Eileen M. Russo & Richard S. Allen, 1998. "CONFIDE: A Collective Decision-Making Procedure Using Confidence Estimates of Individual Judgements," Group Decision and Negotiation, Springer, vol. 7(2), pages 179-194, March.
    5. Alison Hubbard Ashton & Robert H. Ashton, 1985. "Aggregating Subjective Forecasts: Some Empirical Results," Management Science, INFORMS, vol. 31(12), pages 1499-1508, December.
    6. Yaniv, Ilan, 1997. "Weighting and Trimming: Heuristics for Aggregating Judgments under Uncertainty," Organizational Behavior and Human Decision Processes, Elsevier, vol. 69(3), pages 237-249, March.
    7. Camerer, Colin F, 1987. "Do Biases in Probability Judgment Matter in Markets? Experimental Evidence," American Economic Review, American Economic Association, vol. 77(5), pages 981-997, December.
    8. Fischer, Ilan & Budescu, David V., 2005. "When do those who know more also know more about how much they know? The development of confidence and performance in categorical decision tasks," Organizational Behavior and Human Decision Processes, Elsevier, vol. 98(1), pages 39-53, September.
    9. Yates, J. Frank & Lee, Ju-Whei & Shinotsuka, Hiromi & Patalano, Andrea L. & Sieck, Winston R., 1998. "Cross-Cultural Variations in Probability Judgment Accuracy: Beyond General Knowledge Overconfidence?," Organizational Behavior and Human Decision Processes, Elsevier, vol. 74(2), pages 89-117, May.
    10. Robert T. Clemen & Robert L. Winkler, 1990. "Unanimity and Compromise Among Probability Forecasters," Management Science, INFORMS, vol. 36(7), pages 767-779, July.
    11. Cade Massey & George Wu, 2005. "Detecting Regime Shifts: The Causes of Under- and Overreaction," Management Science, INFORMS, vol. 51(6), pages 932-947, June.
    12. Stephen C. Hora, 2004. "Probability Judgments for Continuous Quantities: Linear Combinations and Calibration," Management Science, INFORMS, vol. 50(5), pages 597-604, May.
    13. Clemen, Robert T., 1989. "Combining forecasts: A review and annotated bibliography," International Journal of Forecasting, Elsevier, vol. 5(4), pages 559-583.
    14. Fischer, Ilan & Harvey, Nigel, 1999. "Combining forecasts: What information do judges need to outperform the simple average?," International Journal of Forecasting, Elsevier, vol. 15(3), pages 227-246, July.
    15. Robert L. Winkler & Roy M. Poses, 1993. "Evaluating and Combining Physicians' Probabilities of Survival in an Intensive Care Unit," Management Science, INFORMS, vol. 39(12), pages 1526-1543, December.
    16. Robert F. Bordley, 1982. "A Multiplicative Formula for Aggregating Probability Assessments," Management Science, INFORMS, vol. 28(10), pages 1137-1148, October.
    17. Jason Dana & Robyn M. Dawes, 2004. "The Superiority of Simple Alternatives to Regression for Social Science Predictions," Journal of Educational and Behavioral Statistics, , vol. 29(3), pages 317-331, September.
    18. Daniel Zizzo, 2003. "Verbal and Behavioral Learning in a Probability Compounding Task," Theory and Decision, Springer, vol. 54(4), pages 287-314, June.
    19. Sniezek, Janet A. & Henry, Rebecca A., 1989. "Accuracy and confidence in group judgment," Organizational Behavior and Human Decision Processes, Elsevier, vol. 43(1), pages 1-28, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Peter Bednarik & Thomas Schultze, 2015. "The effectiveness of imperfect weighting in advice taking," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 10(3), pages 265-276, May.
    2. L. Robin Keller, 2009. "From the Editor..," Decision Analysis, INFORMS, vol. 6(1), pages 1-3, March.
    3. L. Robin Keller & Ali Abbas & Manel Baucells & Vicki M. Bier & David Budescu & John C. Butler & Philippe Delquié & Jason R. W. Merrick & Ahti Salo & George Wu, 2010. "From the Editors..," Decision Analysis, INFORMS, vol. 7(4), pages 327-330, December.
      • L. Robin Keller & Manel Baucells & Kevin F. McCardle & Gregory S. Parnell & Ahti Salo, 2007. "From the Editors..," Decision Analysis, INFORMS, vol. 4(4), pages 173-175, December.
      • L. Robin Keller & Manel Baucells & John C. Butler & Philippe Delquié & Jason R. W. Merrick & Gregory S. Parnell & Ahti Salo, 2008. "From the Editors..," Decision Analysis, INFORMS, vol. 5(4), pages 173-176, December.
      • L. Robin Keller & Manel Baucells & John C. Butler & Philippe Delquié & Jason R. W. Merrick & Gregory S. Parnell & Ahti Salo, 2009. "From the Editors ..," Decision Analysis, INFORMS, vol. 6(4), pages 199-201, December.
    4. Stefan Palan & Jürgen Huber & Larissa Senninger, 2020. "Aggregation mechanisms for crowd predictions," Experimental Economics, Springer;Economic Science Association, vol. 23(3), pages 788-814, September.
    5. repec:cup:judgdm:v:10:y:2015:i:3:p:265-276 is not listed on IDEAS
    6. Aurélien Baillon & Laure Cabantous & Peter Wakker, 2012. "Aggregating imprecise or conflicting beliefs: An experimental investigation using modern ambiguity theories," Journal of Risk and Uncertainty, Springer, vol. 44(2), pages 115-147, April.
    7. L. Robin Keller & Ali Abbas & J. Eric Bickel & Vicki M. Bier & David V. Budescu & John C. Butler & Philippe Delquié & Kenneth C. Lichtendahl & Jason R. W. Merrick & Ahti Salo & George Wu, 2011. "From the Editors ---Probability Scoring Rules, Ambiguity, Multiattribute Terrorist Utility, and Sensitivity Analysis," Decision Analysis, INFORMS, vol. 8(4), pages 251-255, December.
    8. Robert Mislavsky & Celia Gaertig, 2022. "Combining Probability Forecasts: 60% and 60% Is 60%, but Likely and Likely Is Very Likely," Management Science, INFORMS, vol. 68(1), pages 541-563, January.
    9. Ilan Yaniv & Shoham Choshen-Hillel & Maxim Milyavsky, 2008. "Spurious Consensus and Opinion Revision: Why Might People Be More Confident in Their Less Accurate Judgments?," Discussion Paper Series dp492, The Federmann Center for the Study of Rationality, the Hebrew University, Jerusalem.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Budescu, David V. & Rantilla, Adrian K. & Yu, Hsiu-Ting & Karelitz, Tzur M., 2003. "The effects of asymmetry among advisors on the aggregation of their opinions," Organizational Behavior and Human Decision Processes, Elsevier, vol. 90(1), pages 178-194, January.
    2. Yaniv, Ilan, 2004. "Receiving other people's advice: Influence and benefit," Organizational Behavior and Human Decision Processes, Elsevier, vol. 93(1), pages 1-13, January.
    3. Soll, Jack B. & Mannes, Albert E., 2011. "Judgmental aggregation strategies depend on whether the self is involved," International Journal of Forecasting, Elsevier, vol. 27(1), pages 81-102.
    4. Soll, Jack B. & Mannes, Albert E., 2011. "Judgmental aggregation strategies depend on whether the self is involved," International Journal of Forecasting, Elsevier, vol. 27(1), pages 81-102, January.
    5. Yaniv, Ilan & Milyavsky, Maxim, 2007. "Using advice from multiple sources to revise and improve judgments," Organizational Behavior and Human Decision Processes, Elsevier, vol. 103(1), pages 104-120, May.
    6. Robert T. Clemen & Robert L. Winkler, 1999. "Combining Probability Distributions From Experts in Risk Analysis," Risk Analysis, John Wiley & Sons, vol. 19(2), pages 187-203, April.
    7. Yaniv, Ilan & Kleinberger, Eli, 2000. "Advice Taking in Decision Making: Egocentric Discounting and Reputation Formation," Organizational Behavior and Human Decision Processes, Elsevier, vol. 83(2), pages 260-281, November.
    8. Bonaccio, Silvia & Dalal, Reeshad S., 2006. "Advice taking and decision-making: An integrative literature review, and implications for the organizational sciences," Organizational Behavior and Human Decision Processes, Elsevier, vol. 101(2), pages 127-151, November.
    9. Albert E. Mannes, 2009. "Are We Wise About the Wisdom of Crowds? The Use of Group Judgments in Belief Revision," Management Science, INFORMS, vol. 55(8), pages 1267-1279, August.
    10. James E. Smith & Detlof von Winterfeldt, 2004. "Anniversary Article: Decision Analysis in Management Science," Management Science, INFORMS, vol. 50(5), pages 561-574, May.
    11. Wang, Xiaoqian & Hyndman, Rob J. & Li, Feng & Kang, Yanfei, 2023. "Forecast combinations: An over 50-year review," International Journal of Forecasting, Elsevier, vol. 39(4), pages 1518-1547.
    12. Van Swol, Lyn M., 2011. "Forecasting another’s enjoyment versus giving the right answer: Trust, shared values, task effects, and confidence in improving the acceptance of advice," International Journal of Forecasting, Elsevier, vol. 27(1), pages 103-120.
    13. repec:cup:judgdm:v:10:y:2015:i:3:p:265-276 is not listed on IDEAS
    14. Hurley, W. J. & Lior, D. U., 2002. "Combining expert judgment: On the performance of trimmed mean vote aggregation procedures in the presence of strategic voting," European Journal of Operational Research, Elsevier, vol. 140(1), pages 142-147, July.
    15. Anil Gaba & Ilia Tsetlin & Robert L. Winkler, 2017. "Combining Interval Forecasts," Decision Analysis, INFORMS, vol. 14(1), pages 1-20, March.
    16. Lyon, Aidan & Wintle, Bonnie C. & Burgman, Mark, 2015. "Collective wisdom: Methods of confidence interval aggregation," Journal of Business Research, Elsevier, vol. 68(8), pages 1759-1767.
    17. Ilan Yaniv, 2005. "Receiving Other People's Advice: Influence and Benefit," Discussion Paper Series dp405, The Federmann Center for the Study of Rationality, the Hebrew University, Jerusalem.
    18. Van Swol, Lyn M., 2011. "Forecasting another's enjoyment versus giving the right answer: Trust, shared values, task effects, and confidence in improving the acceptance of advice," International Journal of Forecasting, Elsevier, vol. 27(1), pages 103-120, January.
    19. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    20. Thomson, Mary E. & Pollock, Andrew C. & Önkal, Dilek & Gönül, M. Sinan, 2019. "Combining forecasts: Performance and coherence," International Journal of Forecasting, Elsevier, vol. 35(2), pages 474-484.
    21. von der Gracht, Heiko A. & Hommel, Ulrich & Prokesch, Tobias & Wohlenberg, Holger, 2016. "Testing weighting approaches for forecasting in a Group Wisdom Support System environment," Journal of Business Research, Elsevier, vol. 69(10), pages 4081-4094.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ordeca:v:3:y:2006:i:3:p:145-162. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.