IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2607.10460.html

Learning from an Unknown DGP: Experimental Evidence on Belief Updating with AI Recommendations

Author

Listed:
  • Matthew Kovach
  • Daniel Martin
  • Gerelt Tserenjigmid

Abstract

We use a controlled experiment to study how beliefs are updated after receiving qualitative information (AI recommendations) from an unknown data-generating process (DGP). Across 60,252 pairs of prior and posterior beliefs, we document three behavioral patterns: updates close to zero when recommendations confirm extreme priors, larger updates when recommendations contradict extreme priors, and smaller updates for intermediate priors. These three behavioral patterns suggest four testable properties of belief updating, which we assess at the aggregate and individual levels. Finally, we examine how well updates are captured by three models of belief updating.

Suggested Citation

  • Matthew Kovach & Daniel Martin & Gerelt Tserenjigmid, 2026. "Learning from an Unknown DGP: Experimental Evidence on Belief Updating with AI Recommendations," Papers 2607.10460, arXiv.org.
  • Handle: RePEc:arx:papers:2607.10460
    as

    Download full text from publisher

    File URL: https://arxiv.org/pdf/2607.10460
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Drew Fudenberg & Jon Kleinberg & Annie Liang & Sendhil Mullainathan, 2022. "Measuring the Completeness of Economic Models," Journal of Political Economy, University of Chicago Press, vol. 130(4), pages 956-990.
    2. Andrew Caplin & Daniel Martin, 2015. "A Testable Theory of Imperfect Perception," Economic Journal, Royal Economic Society, vol. 125(582), pages 184-202, February.
    3. Evan Sadler, 2021. "A Practical Guide to Updating Beliefs From Contradictory Evidence," Econometrica, Econometric Society, vol. 89(1), pages 415-436, January.
    4. Kathleen Ngangoué, M., 2021. "Learning under ambiguity: An experiment in gradual information processing," Journal of Economic Theory, Elsevier, vol. 195(C).
    5. Cohen, M. & Gilboa, I. & Jaffray, J.Y. & Schmeidler, D., 2000. "An experimental study of updating ambiguous beliefs," Risk, Decision and Policy, Cambridge University Press, vol. 5(2), pages 123-133, June.
    6. Jaden Yang Chen, 2026. "Sequential Learning under Informational Ambiguity," American Economic Review, American Economic Association, vol. 116(1), pages 209-245, January.
    7. Gary Charness & Ryan Oprea & Sevgi Yuksel, 2021. "How do People Choose Between Biased Information Sources? Evidence from a Laboratory Experiment," Journal of the European Economic Association, European Economic Association, vol. 19(3), pages 1656-1691.
    8. Larry G Epstein & Yoram Halevy, 2024. "Hard-to-Interpret Signals," Journal of the European Economic Association, European Economic Association, vol. 22(1), pages 393-427.
    9. Ke, Shaowei & Wu, Brian & Zhao, Chen, 2024. "Learning from a black box," Journal of Economic Theory, Elsevier, vol. 221(C).
    10. David M. Grether, 1980. "Bayes Rule as a Descriptive Model: The Representativeness Heuristic," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 95(3), pages 537-557.
    11. Ignacio Esponda & Emanuel Vespa & Sevgi Yuksel, 2024. "Mental Models and Learning: The Case of Base-Rate Neglect," American Economic Review, American Economic Association, vol. 114(3), pages 752-782, March.
    12. De Filippis, Roberta & Guarino, Antonio & Jehiel, Philippe & Kitagawa, Toru, 2022. "Non-Bayesian updating in a social learning experiment," Journal of Economic Theory, Elsevier, vol. 199(C).
    13. Fabian Stephany & Jedrzej Duszynski, 2026. "Women Worry, Men Adopt? Gendered Risk Perceptions and Generative AI Adoption," Papers 2601.03880, arXiv.org, revised Jul 2026.
    14. Holt, Charles A. & Smith, Angela M., 2009. "An update on Bayesian updating," Journal of Economic Behavior & Organization, Elsevier, vol. 69(2), pages 125-134, February.
    15. Adrian Bruhin & Helga Fehr-Duda & Thomas Epper, 2010. "Risk and Rationality: Uncovering Heterogeneity in Probability Distortion," Econometrica, Econometric Society, vol. 78(4), pages 1375-1412, July.
    16. Eran Shmaya & Leeat Yariv, 2016. "Experiments on Decisions under Uncertainty: A Theoretical Framework," American Economic Review, American Economic Association, vol. 106(7), pages 1775-1801, July.
    17. Shishkin, Denis & Ortoleva, Pietro, 2023. "Ambiguous information and dilation: An experiment," Journal of Economic Theory, Elsevier, vol. 208(C).
    18. Matthew Rabin, 2002. "Inference by Believers in the Law of Small Numbers," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 117(3), pages 775-816.
    19. Simone Cerreia-Vioglio & Roberto Corrao & Giacomo Lanzani, 2024. "Dynamic Opinion Aggregation: Long-Run Stability and Disagreement," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 91(3), pages 1406-1447.
    20. Zhao, Chen, 2022. "Pseudo-Bayesian updating," Theoretical Economics, Econometric Society, vol. 17(1), January.
    21. Sniezek, Janet A. & Buckley, Timothy, 1995. "Cueing and Cognitive Conflict in Judge-Advisor Decision Making," Organizational Behavior and Human Decision Processes, Elsevier, vol. 62(2), pages 159-174, May.
    22. Alexander Coutts, 2019. "Good news and bad news are still news: experimental evidence on belief updating," Experimental Economics, Springer;Economic Science Association, vol. 22(2), pages 369-395, June.
    23. Pietro Ortoleva, 2012. "Modeling the Change of Paradigm: Non-Bayesian Reactions to Unexpected News," American Economic Review, American Economic Association, vol. 102(6), pages 2410-2436, October.
    24. repec:hal:pseose:halshs-01155313 is not listed on IDEAS
    25. Tanjim Hossain & Ryo Okui, 2013. "The Binarized Scoring Rule," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 80(3), pages 984-1001.
    26. Roland G FryerJr & Philipp Harms & Matthew O Jackson, 2019. "Updating Beliefs when Evidence is Open to Interpretation: Implications for Bias and Polarization," Journal of the European Economic Association, European Economic Association, vol. 17(5), pages 1470-1501.
    27. David Danz & Lise Vesterlund & Alistair J. Wilson, 2022. "Belief Elicitation and Behavioral Incentive Compatibility," American Economic Review, American Economic Association, vol. 112(9), pages 2851-2883, September.
    28. Kovach, Matthew, 2020. "Twisting the truth: foundations of wishful thinking," Theoretical Economics, Econometric Society, vol. 15(3), July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Elchin Suleymanov, 2025. "Robust Maximum Likelihood Updating," Papers 2504.17151, arXiv.org, revised Dec 2025.
    2. Castagnetti, Alessandro & Schmacker, Renke, 2022. "Protecting the ego: Motivated information selection and updating," European Economic Review, Elsevier, vol. 142(C).
    3. J. Aislinn Bohren & Daniel N. Hauser, 2023. "Behavioral Foundations of Model Misspecification," PIER Working Paper Archive 23-007, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania.
    4. Yucheng Liang, 2025. "Learning from Unknown Information Sources," Management Science, INFORMS, vol. 71(5), pages 3873-3890, May.
    5. Yanagita, Shohei, 2025. "Overreacting to a black box," Journal of Mathematical Economics, Elsevier, vol. 118(C).
    6. Ilke Aydogan & Aurélien Baillon & Emmanuel Kemel & Chen Li, 2025. "How much do we learn? Measuring symmetric and asymmetric deviations from Bayesian updating through choices," Quantitative Economics, Econometric Society, vol. 16(1), pages 329-365, January.
    7. Minnich, Aljoscha & Roggenkamp, Hauke & Lange, Andreas, 2024. "Ambiguity attitudes and surprises: Experimental evidence on communicating new information within a large population sample," Journal of Economic Behavior & Organization, Elsevier, vol. 228(C).
    8. Kai Barron, 2021. "Belief updating: does the ‘good-news, bad-news’ asymmetry extend to purely financial domains?," Experimental Economics, Springer;Economic Science Association, vol. 24(1), pages 31-58, March.
    9. Cheng, Ing-Haw & Hsiaw, Alice, 2022. "Distrust in experts and the origins of disagreement," Journal of Economic Theory, Elsevier, vol. 200(C).
    10. Ke, Shaowei & Wu, Brian & Zhao, Chen, 2024. "Learning from a black box," Journal of Economic Theory, Elsevier, vol. 221(C).
    11. Wenbo Zou & Xue Xu, 2023. "Ingroup bias in a social learning experiment," Experimental Economics, Springer;Economic Science Association, vol. 26(1), pages 27-54, March.
    12. Ro’i Zultan & Aniol Llorente-Saguer & Santiago Oliveros, 2024. "Beyond Value: on the Role of Symmetryin Demand for Information," Working Papers 2411, Ben-Gurion University of the Negev, Department of Economics.
    13. Ignacio Esponda & Emanuel Vespa & Sevgi Yuksel, 2024. "Mental Models and Learning: The Case of Base-Rate Neglect," American Economic Review, American Economic Association, vol. 114(3), pages 752-782, March.
    14. Markus M. Möbius & Muriel Niederle & Paul Niehaus & Tanya S. Rosenblat, 2022. "Managing Self-Confidence: Theory and Experimental Evidence," Management Science, INFORMS, vol. 68(11), pages 7793-7817, November.
    15. James Bland & Yaroslav Rosokha, 2024. "Rounding the (Non)Bayesian Curve: Unraveling the Effects of Rounding Errors in Belief Updating," Purdue University Economics Working Papers 1353, Purdue University, Department of Economics.
    16. Kathleen Ngangoué, M., 2021. "Learning under ambiguity: An experiment in gradual information processing," Journal of Economic Theory, Elsevier, vol. 195(C).
    17. Daniel J. Benjamin, 2018. "Errors in Probabilistic Reasoning and Judgment Biases," GRU Working Paper Series GRU_2018_023, City University of Hong Kong, Department of Economics and Finance, Global Research Unit.
    18. Charlotte Cordes & Jana Friedrichsen & Simeon Schudy, 2025. "Motivated Beliefs Under Delayed Uncertainty Resolution," CESifo Working Paper Series 12286, CESifo.
    19. Anwesha Banerjee & Katharina Momsen, 2025. "(Un-)scientifically Spun: Narratives, Belief Updating, and Pro-Environmental Behavior," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 88(12), pages 3873-3903, December.
    20. Kenneth Chan & Gary Charness & Chetan Dave & J. Lucas Reddinger, 2024. "On Prior Confidence and Belief Updating," Working Papers 2024-10, University of Alberta, Department of Economics.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2607.10460. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: https://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.