IDEAS home Printed from https://ideas.repec.org/a/inm/ormnsc/v68y2022i1p541-563.html
   My bibliography  Save this article

Combining Probability Forecasts: 60% and 60% Is 60%, but Likely and Likely Is Very Likely

Author

Listed:
  • Robert Mislavsky

    (Carey Business School, Johns Hopkins University, Baltimore, Maryland 21202)

  • Celia Gaertig

    (Booth School of Business, University of Chicago, Chicago, Illinois 60637)

Abstract

How do we combine others’ probability forecasts? Prior research has shown that when advisors provide numeric probability forecasts, people typically average them (i.e., they move closer to the average advisor’s forecast). However, what if the advisors say that an event is “likely” or “probable?” In eight studies ( n = 7,334), we find that people are more likely to act as if they “count” verbal probabilities (i.e., they move closer to certainty than any individual advisor’s forecast) than they are to “count” numeric probabilities. For example, when the advisors both say an event is “likely,” participants will say that it is “very likely.” This effect occurs for both probabilities above and below 50%, for hypothetical scenarios and real events, and when presenting the others’ forecasts simultaneously or sequentially. We also show that this combination strategy carries over to subsequent consumer decisions that rely on advisors’ likelihood judgments. We discuss and rule out several candidate mechanisms for our effect.

Suggested Citation

  • Robert Mislavsky & Celia Gaertig, 2022. "Combining Probability Forecasts: 60% and 60% Is 60%, but Likely and Likely Is Very Likely," Management Science, INFORMS, vol. 68(1), pages 541-563, January.
  • Handle: RePEc:inm:ormnsc:v:68:y:2022:i:1:p:541-563
    DOI: 10.1287/mnsc.2020.3902
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/mnsc.2020.3902
    Download Restriction: no

    File URL: https://libkey.io/10.1287/mnsc.2020.3902?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Daniel Kahneman & Dan Lovallo, 1993. "Timid Choices and Bold Forecasts: A Cognitive Perspective on Risk Taking," Management Science, INFORMS, vol. 39(1), pages 17-31, January.
    2. Richard P. Larrick & Jack B. Soll, 2006. "Erratum--Intuitions About Combining Opinions: Misappreciation of the Averaging Principle," Management Science, INFORMS, vol. 52(2), pages 309-310, February.
    3. Teigen, Karl Halvor & Brun, Wibecke, 1999. "The Directionality of Verbal Probability Expressions: Effects on Decisions, Predictions, and Probabilistic Reasoning, , , ," Organizational Behavior and Human Decision Processes, Elsevier, vol. 80(2), pages 155-190, November.
    4. Dipayan Biswas & Guangzhi Zhao & Donald R. Lehmann, 2011. "The Impact of Sequential Data on Consumer Confidence in Relative Judgments," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 37(5), pages 874-887.
    5. Sah, Sunita & Loewenstein, George, 2015. "Conflicted advice and second opinions: Benefits, but unintended consequences," Organizational Behavior and Human Decision Processes, Elsevier, vol. 130(C), pages 89-107.
    6. Alison Hubbard Ashton & Robert H. Ashton, 1985. "Aggregating Subjective Forecasts: Some Empirical Results," Management Science, INFORMS, vol. 31(12), pages 1499-1508, December.
    7. West, Patricia M & Broniarczyk, Susan M, 1998. "Integrating Multiple Opinions: The Role of Aspiration Level on Consumer Response to Critic Consensus," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 25(1), pages 38-51, June.
    8. Jonathan Baron & Barbara A. Mellers & Philip E. Tetlock & Eric Stone & Lyle H. Ungar, 2014. "Two Reasons to Make Aggregated Probability Forecasts More Extreme," Decision Analysis, INFORMS, vol. 11(2), pages 133-145, June.
    9. Wallsten, Thomas S. & Diederich, Adele, 2001. "Understanding pooled subjective probability estimates," Mathematical Social Sciences, Elsevier, vol. 41(1), pages 1-18, January.
    10. Yaniv, Ilan & Milyavsky, Maxim, 2007. "Using advice from multiple sources to revise and improve judgments," Organizational Behavior and Human Decision Processes, Elsevier, vol. 103(1), pages 104-120, May.
    11. David V. Budescu & Hsiu-Ting Yu, 2006. "To Bayes or Not to Bayes? A Comparison of Two Classes of Models of Information Aggregation," Decision Analysis, INFORMS, vol. 3(3), pages 145-162, September.
    12. Michael Mastrandrea & Katharine Mach & Gian-Kasper Plattner & Ottmar Edenhofer & Thomas Stocker & Christopher Field & Kristie Ebi & Patrick Matschoss, 2011. "The IPCC AR5 guidance note on consistent treatment of uncertainties: a common approach across the working groups," Climatic Change, Springer, vol. 108(4), pages 675-691, October.
    13. Richard P. Larrick & Jack B. Soll, 2006. "Intuitions About Combining Opinions: Misappreciation of the Averaging Principle," Management Science, INFORMS, vol. 52(1), pages 111-127, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Johannes Müller-Trede & Shoham Choshen-Hillel & Meir Barneron & Ilan Yaniv, 2018. "The Wisdom of Crowds in Matters of Taste," Management Science, INFORMS, vol. 64(4), pages 1779-1803, April.
    2. repec:cup:judgdm:v:10:y:2015:i:3:p:265-276 is not listed on IDEAS
    3. Yaniv, Ilan & Choshen-Hillel, Shoham & Milyavsky, Maxim, 2011. "Receiving advice on matters of taste: Similarity, majority influence, and taste discrimination," Organizational Behavior and Human Decision Processes, Elsevier, vol. 115(1), pages 111-120, May.
    4. Don A. Moore & Samuel A. Swift & Angela Minster & Barbara Mellers & Lyle Ungar & Philip Tetlock & Heather H. J. Yang & Elizabeth R. Tenney, 2017. "Confidence Calibration in a Multiyear Geopolitical Forecasting Competition," Management Science, INFORMS, vol. 63(11), pages 3552-3565, November.
    5. Peter Bednarik & Thomas Schultze, 2015. "The effectiveness of imperfect weighting in advice taking," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 10(3), pages 265-276, May.
    6. repec:cup:judgdm:v:14:y:2019:i:4:p:395-411 is not listed on IDEAS
    7. Patrick Afflerbach & Christopher Dun & Henner Gimpel & Dominik Parak & Johannes Seyfried, 2021. "A Simulation-Based Approach to Understanding the Wisdom of Crowds Phenomenon in Aggregating Expert Judgment," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 63(4), pages 329-348, August.
    8. Alison Wood Brooks & Francesca Gino & Maurice E. Schweitzer, 2015. "Smart People Ask for (My) Advice: Seeking Advice Boosts Perceptions of Competence," Management Science, INFORMS, vol. 61(6), pages 1421-1435, June.
    9. Ilan Yaniv & Shoham Choshen-Hillel, 2012. "When guessing what another person would say is better than giving your own opinion: Using perspective-taking to improve advice-taking," Discussion Paper Series dp622, The Federmann Center for the Study of Rationality, the Hebrew University, Jerusalem.
    10. Albert E. Mannes, 2009. "Are We Wise About the Wisdom of Crowds? The Use of Group Judgments in Belief Revision," Management Science, INFORMS, vol. 55(8), pages 1267-1279, August.
    11. Saemi Park & David V. Budescu, 2015. "Aggregating multiple probability intervals to improve calibration," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 10(2), pages 130-143, March.
    12. Ying Han & David Budescu, 2019. "A universal method for evaluating the quality of aggregators," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 14(4), pages 395-411, July.
    13. repec:cup:judgdm:v:10:y:2015:i:2:p:130-143 is not listed on IDEAS
    14. Jerker Denrell & Christina Fang, 2010. "Predicting the Next Big Thing: Success as a Signal of Poor Judgment," Management Science, INFORMS, vol. 56(10), pages 1653-1667, October.
    15. Aurélien Baillon & Laure Cabantous & Peter Wakker, 2012. "Aggregating imprecise or conflicting beliefs: An experimental investigation using modern ambiguity theories," Journal of Risk and Uncertainty, Springer, vol. 44(2), pages 115-147, April.
    16. repec:cup:judgdm:v:11:y:2016:i:4:p:401-415 is not listed on IDEAS
    17. Soll, Jack B. & Mannes, Albert E., 2011. "Judgmental aggregation strategies depend on whether the self is involved," International Journal of Forecasting, Elsevier, vol. 27(1), pages 81-102.
    18. Johannes Müller-Trede & Shoham Choshen-Hillel & Meir Barneron & Ilan Yaniv, 2017. "The Wisdom of Crowds in Matters of Taste," Discussion Paper Series dp709, The Federmann Center for the Study of Rationality, the Hebrew University, Jerusalem.
    19. Soll, Jack B. & Mannes, Albert E., 2011. "Judgmental aggregation strategies depend on whether the self is involved," International Journal of Forecasting, Elsevier, vol. 27(1), pages 81-102, January.
    20. Asa B. Palley & Jack B. Soll, 2019. "Extracting the Wisdom of Crowds When Information Is Shared," Management Science, INFORMS, vol. 67(5), pages 2291-2309, May.
    21. Yaniv, Ilan & Milyavsky, Maxim, 2007. "Using advice from multiple sources to revise and improve judgments," Organizational Behavior and Human Decision Processes, Elsevier, vol. 103(1), pages 104-120, May.
    22. von der Gracht, Heiko A. & Hommel, Ulrich & Prokesch, Tobias & Wohlenberg, Holger, 2016. "Testing weighting approaches for forecasting in a Group Wisdom Support System environment," Journal of Business Research, Elsevier, vol. 69(10), pages 4081-4094.
    23. Mandy Hütter & Fabian Ache, 2016. "Seeking advice: A sampling approach to advice taking," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 11(4), pages 401-415, July.
    24. Boris Maciejovsky & David V. Budescu, 2020. "Too Much Trust in Group Decisions: Uncovering Hidden Profiles by Groups and Markets," Organization Science, INFORMS, vol. 31(6), pages 1497-1514, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormnsc:v:68:y:2022:i:1:p:541-563. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.