IDEAS home Printed from https://ideas.repec.org/p/ecl/harjfk/16-016.html
   My bibliography  Save this paper

How Quantifying Probability Assessments Influences Analysis and Decision Making: Experimental Evidence from National Security Professionals

Author

Listed:
  • Friedman, Jeffrey A.

    (Dartmouth College)

  • Lerner, Jennifer S.

    (Harvard University)

  • Zeckhauser, Richard

    (Harvard University)

Abstract

National security is one of many fields where public officials offer imprecise probability assessments when evaluating high-stakes decisions. This practice is often justified with arguments about how quantifying subjective judgments would bias analysts and decision makers toward overconfident action. We translate these arguments into testable hypotheses, and evaluate their validity through survey experiments involving national security professionals. Results reveal that when decision makers receive numerals (as opposed to words) for probability assessments, they are less likely to support risky actions and more receptive to gathering additional information, disconfirming the idea of a bias toward action. Yet when respondents generate probabilities themselves, using numbers (as opposed to words) magnifies overconfidence, especially among low-performing assessors. These results hone directions for research among both proponents and skeptics of quantifying probability estimates in national security and other fields. Given that uncertainty surrounds virtually all intelligence reports, military plans, and national security decisions, understanding how national security officials form and interpret probability assessments has wide-ranging implications for theory and practice.

Suggested Citation

  • Friedman, Jeffrey A. & Lerner, Jennifer S. & Zeckhauser, Richard, 2015. "How Quantifying Probability Assessments Influences Analysis and Decision Making: Experimental Evidence from National Security Professionals," Working Paper Series 16-016, Harvard University, John F. Kennedy School of Government.
    • Handle: RePEc:ecl:harjfk:16-016
    as

    Download full text from publisher

    File URL: https://research.hks.harvard.edu/publications/getFile.aspx?Id=1383
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Piercey, M. David, 2009. "Motivated reasoning and verbal vs. numerical probability assessment: Evidence from an accounting context," Organizational Behavior and Human Decision Processes, Elsevier, vol. 108(2), pages 330-341, March.
    2. Press, Daryl G. & Sagan, Scott D. & Valentino, Benjamin A., 2013. "Atomic Aversion: Experimental Evidence on Taboos, Traditions, and the Non-Use of Nuclear Weapons," American Political Science Review, Cambridge University Press, vol. 107(1), pages 188-206, February.
    3. David V. Budescu & Han-Hui Por & Stephen B. Broomell & Michael Smithson, 2014. "The interpretation of IPCC probabilistic statements around the world," Nature Climate Change, Nature, vol. 4(6), pages 508-512, June.
    4. repec:cup:judgdm:v:7:y:2012:i:1:p:25-47 is not listed on IDEAS
    5. Tomz, Michael R. & Weeks, Jessica L. P., 2013. "Public Opinion and the Democratic Peace," American Political Science Review, Cambridge University Press, vol. 107(4), pages 849-865, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. repec:cup:judgdm:v:14:y:2019:i:6:p:683-695 is not listed on IDEAS
    2. Dawn Liu Holford & Marie Juanchich & Tom Foulsham & Miroslav Sirota & Alasdair D. F. Clarke, 2021. "Eye-tracking evidence for fixation asymmetries in verbal and numerical quantifier processing," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 16(4), pages 969-1009, July.
    3. repec:cup:judgdm:v:16:y:2021:i:4:p:969-1009 is not listed on IDEAS
    4. Robert N. Collins & David R. Mandel, 2019. "Cultivating credibility with probability words and numbers," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 14(6), pages 683-695, November.
    5. David R. Mandel & Daniel Irwin, 2021. "Tracking accuracy of strategic intelligence forecasts: Findings from a long‐term Canadian study," Futures & Foresight Science, John Wiley & Sons, vol. 3(3-4), September.
    6. Daniel L. Nielson & Susan D. Hyde & Judith Kelley, 2019. "The elusive sources of legitimacy beliefs: Civil society views of international election observers," The Review of International Organizations, Springer, vol. 14(4), pages 685-715, December.
    7. Xiaojun Li & Dingding Chen, 2021. "Public opinion, international reputation, and audience costs in an authoritarian regime," Conflict Management and Peace Science, Peace Science Society (International), vol. 38(5), pages 543-560, September.
    8. P A Hancock & William G Volante, 2020. "Quantifying the qualities of language," PLOS ONE, Public Library of Science, vol. 15(5), pages 1-27, May.
    9. Heather P. Lacey & Steven C. Lacey & Prerna Dayal & Caroline Forest & Dana Blasi, 2023. "Context Matters: Emotional Sensitivity to Probabilities and the Bias for Action in Cancer Treatment Decisions," Medical Decision Making, , vol. 43(4), pages 417-429, May.
    10. Jonathan A. Chu, 2019. "A Clash of Norms? How Reciprocity and International Humanitarian Law affect American Opinion on the Treatment of POWs," Journal of Conflict Resolution, Peace Science Society (International), vol. 63(5), pages 1140-1164, May.
    11. Zhaobo Chen & Gangzhu Qiao & Jianchao Zeng, 2019. "Study on the Relationship between Worker States and Unsafe Behaviours in Coal Mine Accidents Based on a Bayesian Networks Model," Sustainability, MDPI, vol. 11(18), pages 1-16, September.
    12. Christopher W. Blair & Jonathan A. Chu & Joshua A. Schwartz, 2022. "The Two Faces of Opposition to Chemical Weapons: Sincere Versus Insincere Norm-Holders," Journal of Conflict Resolution, Peace Science Society (International), vol. 66(4-5), pages 677-703, May.
    13. Kuselias, Stephen & Agoglia, Christopher P. & Wang, Elaine Ying, 2023. "The effect of team member proximity and assignment length on audit staff reliance on a supervisor's preferences," Accounting, Organizations and Society, Elsevier, vol. 105(C).
    14. Michal Onderco & Michal Smetana & Tom W. Etienne, 2023. "Hawks in the making? European public views on nuclear weapons post‐Ukraine," Global Policy, London School of Economics and Political Science, vol. 14(2), pages 305-317, May.
    15. repec:cup:judgdm:v:16:y:2021:i:2:p:363-393 is not listed on IDEAS
    16. Bernauer, Thomas & Spilker, Gabriele & Umaña, Víctor, 2014. "Different countries same partners: Experimental Evidence on PTA Partner Country Choice from Costa Rica, Nicaragua and Vietnam," Papers 739, World Trade Institute.
    17. Michael D. Gerst & Melissa A. Kenney & Irina Feygina, 2021. "Improving the usability of climate indicator visualizations through diagnostic design principles," Climatic Change, Springer, vol. 166(3), pages 1-22, June.
    18. Rudolph, Lukas & Freitag, Markus & Thurner, Paul, 2021. "The Comparative Legitimacy of Arms Exports - A Conjoint Experiment in Germany and France," SocArXiv r73pv, Center for Open Science.
    19. Mike Farjam & Olexandr Nikolaychuk & Giangiacomo Bravo, 2019. "Investing into climate change mitigation despite the risk of failure," Climatic Change, Springer, vol. 154(3), pages 453-460, June.
    20. Robert Gampfer, 2016. "Minilateralism or the UNFCCC? The Political Feasibility of Climate Clubs," Global Environmental Politics, MIT Press, vol. 16(3), pages 62-88, August.
    21. Jessen L. Hobson & Matthew T. Stern & Aaron F. Zimbelman, 2020. "The Benefit of Mean Auditors: The Influence of Social Interaction and the Dark Triad on Unjustified Auditor Trust," Contemporary Accounting Research, John Wiley & Sons, vol. 37(2), pages 1217-1247, June.
    22. Nathan F. Dieckmann & Robin Gregory & Ellen Peters & Robert Hartman, 2017. "Seeing What You Want to See: How Imprecise Uncertainty Ranges Enhance Motivated Reasoning," Risk Analysis, John Wiley & Sons, vol. 37(3), pages 471-486, March.
    23. Leo Tang & Marietta Peytcheva & Pei Li, 2020. "Investor-Paid Ratings and Conflicts of Interest," Journal of Business Ethics, Springer, vol. 163(2), pages 365-378, May.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ecl:harjfk:16-016. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/ksharus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.