IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2305.02823.html
   My bibliography  Save this paper

Surveying Generative AI's Economic Expectations

Author

Listed:
  • Leland Bybee

Abstract

I introduce a survey of economic expectations formed by querying a large language model (LLM)'s expectations of various financial and macroeconomic variables based on a sample of news articles from the Wall Street Journal between 1984 and 2021. I find the resulting expectations closely match existing surveys including the Survey of Professional Forecasters (SPF), the American Association of Individual Investors, and the Duke CFO Survey. Importantly, I document that LLM based expectations match many of the deviations from full-information rational expectations exhibited in these existing survey series. The LLM's macroeconomic expectations exhibit under-reaction commonly found in consensus SPF forecasts. Additionally, its return expectations are extrapolative, disconnected from objective measures of expected returns, and negatively correlated with future realized returns. Finally, using a sample of articles outside of the LLM's training period I find that the correlation with existing survey measures persists -- indicating these results do not reflect memorization but generalization on the part of the LLM. My results provide evidence for the potential of LLMs to help us better understand human beliefs and navigate possible models of nonrational expectations.

Suggested Citation

  • Leland Bybee, 2023. "Surveying Generative AI's Economic Expectations," Papers 2305.02823, arXiv.org, revised May 2023.
  • Handle: RePEc:arx:papers:2305.02823
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2305.02823
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ivo Welch & Amit Goyal, 2008. "A Comprehensive Look at The Empirical Performance of Equity Premium Prediction," The Review of Financial Studies, Society for Financial Studies, vol. 21(4), pages 1455-1508, July.
    2. Pedro Bordalo & Nicola Gennaioli & Yueran Ma & Andrei Shleifer, 2020. "Overreaction in Macroeconomic Expectations," American Economic Review, American Economic Association, vol. 110(9), pages 2748-2782, September.
    3. George-Marios Angeletos & Zhen Huo & Karthik A. Sastry, 2021. "Imperfect Macroeconomic Expectations: Evidence and Theory," NBER Macroeconomics Annual, University of Chicago Press, vol. 35(1), pages 1-86.
    4. Leland Farmer & Emi Nakamura & Jón Steinsson, 2021. "Learning About the Long Run," NBER Working Papers 29495, National Bureau of Economic Research, Inc.
    5. Barberis, Nicholas & Greenwood, Robin & Jin, Lawrence & Shleifer, Andrei, 2015. "X-CAPM: An extrapolative capital asset pricing model," Journal of Financial Economics, Elsevier, vol. 115(1), pages 1-24.
    6. Cutler, David M & Poterba, James M & Summers, Lawrence H, 1990. "Speculative Dynamics and the Role of Feedback Traders," American Economic Review, American Economic Association, vol. 80(2), pages 63-68, May.
    7. John J. Horton, 2023. "Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus?," NBER Working Papers 31122, National Bureau of Economic Research, Inc.
    8. Martin Lettau & Sydney Ludvigson, 2001. "Consumption, Aggregate Wealth, and Expected Stock Returns," Journal of Finance, American Finance Association, vol. 56(3), pages 815-849, June.
    9. Olivier Coibion & Yuriy Gorodnichenko, 2012. "What Can Survey Forecasts Tell Us about Information Rigidities?," Journal of Political Economy, University of Chicago Press, vol. 120(1), pages 116-159.
    10. Pedro Bordalo & Nicola Gennaioli & Andrei Shleifer, 2018. "Diagnostic Expectations and Credit Cycles," Journal of Finance, American Finance Association, vol. 73(1), pages 199-227, February.
    11. Peter Andre & Ingar Haaland & Christopher Roth & Johannes Wohlfart, 2021. "Narratives about the Macroeconomy," ECONtribute Discussion Papers Series 127, University of Bonn and University of Cologne, Germany.
    12. Stefano Giglio & Matteo Maggiori & Johannes Stroebel & Stephen Utkus, 2021. "Five Facts about Beliefs and Portfolios," American Economic Review, American Economic Association, vol. 111(5), pages 1481-1522, May.
    13. Gabaix, Xavier & Koijen, Ralph, 2021. "In Search of the Origins of Financial Fluctuations: The Inelastic Markets Hypothesis," CEPR Discussion Papers 16290, C.E.P.R. Discussion Papers.
    14. Olivier Coibion & Yuriy Gorodnichenko, 2015. "Information Rigidity and the Expectations Formation Process: A Simple Framework and New Facts," American Economic Review, American Economic Association, vol. 105(8), pages 2644-2678, August.
    15. Robert B. Barsky & J. Bradford De Long, 1993. "Why Does the Stock Market Fluctuate?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 108(2), pages 291-311.
    16. Jin, Lawrence J. & Sui, Pengfei, 2022. "Asset pricing with return extrapolation," Journal of Financial Economics, Elsevier, vol. 145(2), pages 273-295.
    17. Sims, Christopher A, 1980. "Macroeconomics and Reality," Econometrica, Econometric Society, vol. 48(1), pages 1-48, January.
    18. Georgij Alekseev & Stefano Giglio & Quinn Maingi & Julia Selgrad & Johannes Stroebel, 2022. "A Quantity-Based Approach to Constructing Climate Risk Hedge Portfolios," NBER Working Papers 30703, National Bureau of Economic Research, Inc.
    19. Hassan Afrouzi & Spencer Y Kwon & Augustin Landier & Yueran Ma & David Thesmar, 2023. "Overreaction in Expectations: Evidence and Theory," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 138(3), pages 1713-1764.
    20. Markus Brunnermeier & Emmanuel Farhi & Ralph S J Koijen & Arvind Krishnamurthy & Sydney C Ludvigson & Hanno Lustig & Stefan Nagel & Monika Piazzesi, 2021. "Review Article: Perspectives on the Future of Asset Pricing [Do survey expectations of stock returns reflect risk-adjustments?]," The Review of Financial Studies, Society for Financial Studies, vol. 34(4), pages 2126-2160.
    21. Lucas, Robert Jr., 1972. "Expectations and the neutrality of money," Journal of Economic Theory, Elsevier, vol. 4(2), pages 103-124, April.
    22. Fama, Eugene F. & French, Kenneth R., 1989. "Business conditions and expected returns on stocks and bonds," Journal of Financial Economics, Elsevier, vol. 25(1), pages 23-49, November.
    23. John J. Horton, 2023. "Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus?," Papers 2301.07543, arXiv.org.
    24. Lars A. Lochstoer & Tyler Muir, 2022. "Volatility Expectations and Returns," Journal of Finance, American Finance Association, vol. 77(2), pages 1055-1096, April.
    25. Lucas, Robert Jr, 1976. "Econometric policy evaluation: A critique," Carnegie-Rochester Conference Series on Public Policy, Elsevier, vol. 1(1), pages 19-46, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Philip Brookins & Jason DeBacker, 2024. "Playing games with GPT: What can we learn about a large language model from canonical strategic games?," Economics Bulletin, AccessEcon, vol. 44(1), pages 25-37.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Born, Benjamin & Enders, Zeno & Menkhoff, Manuel & Müller, Gernot & Niemann, Knut, 2022. "Firm Expectations and News: Micro v Macro," CEPR Discussion Papers 17768, C.E.P.R. Discussion Papers.
    2. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    3. Pedro Bordalo & Nicola Gennaioli & Rafael La Porta & Andrei Shleifer, 2020. "Belief Overreaction and Stock Market Puzzles," NBER Working Papers 27283, National Bureau of Economic Research, Inc.
    4. Conrad, Christian & Lahiri, Kajal, 2023. "Heterogeneous expectations among professional forecasters," ZEW Discussion Papers 23-062, ZEW - Leibniz Centre for European Economic Research.
    5. Zhao Han & Xiaohan Ma & Ruoyun Mao, 2023. "The Role of Dispersed Information in Inflation and Inflation Expectations," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 48, pages 72-106, April.
    6. Leland E. Farmer & Lawrence Schmidt & Allan Timmermann, 2023. "Pockets of Predictability," Journal of Finance, American Finance Association, vol. 78(3), pages 1279-1341, June.
    7. Schmeling, Maik & Schrimpf, Andreas & Steffensen, Sigurd A.M., 2022. "Monetary policy expectation errors," Journal of Financial Economics, Elsevier, vol. 146(3), pages 841-858.
    8. Jean-Paul L'Huillier & Sanjay R. Singh & Donghoon Yoo, 2021. "Incorporating Diagnostic Expectations into the New Keynesian Framework," Working Papers 339, University of California, Davis, Department of Economics.
    9. Peter Andrebriq & Carlo Pizzinelli & Christopher Roth & Johannes Wohlfart, 2022. "Subjective Models of the Macroeconomy: Evidence From Experts and Representative Samples [Rationally Confused: On the Aggregate Implications of Information Provision Policies]," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 89(6), pages 2958-2991.
    10. Nagel, Stefan & Xu, Zhengyang, 2023. "Dynamics of subjective risk premia," Journal of Financial Economics, Elsevier, vol. 150(2).
    11. Hagenhoff, Tim & Lustenhouwer, Joep, 2023. "The role of stickiness, extrapolation and past consensus forecasts in macroeconomic expectations," Journal of Economic Dynamics and Control, Elsevier, vol. 149(C).
    12. Bordalo, Pedro & Gennaioli, Nicola & Kwon, Spencer Yongwook & Shleifer, Andrei, 2021. "Diagnostic bubbles," Journal of Financial Economics, Elsevier, vol. 141(3), pages 1060-1077.
    13. Brückbauer, Frank, 2022. "Do financial market experts know their theory? New evidence from survey data," ZEW Discussion Papers 20-092, ZEW - Leibniz Centre for European Economic Research, revised 2022.
    14. Andrade, Philippe & Gautier, Erwan & Mengus, Eric, 2023. "What matters in households’ inflation expectations?," Journal of Monetary Economics, Elsevier, vol. 138(C), pages 50-68.
    15. Jeffrey Schafer, 2022. "Inflation Expectations and Their Formation: Working Paper 2022-03," Working Papers 57398, Congressional Budget Office.
    16. Lena Draeger & Michael J. Lamla, 2013. "Imperfect information and inflation expectations," KOF Working papers 13-329, KOF Swiss Economic Institute, ETH Zurich.
    17. Florian Peters & Simas Kucinskas, 2018. "Measuring Biases in Expectation Formation," Tinbergen Institute Discussion Papers 18-058/IV, Tinbergen Institute.
    18. von Gaudecker, Hans-Martin & Wogrolly, Axel, 2022. "Heterogeneity in households’ stock market beliefs," Journal of Econometrics, Elsevier, vol. 231(1), pages 232-247.
    19. Shintani, Mototsugu & Ueda, Kozo, 2023. "Identifying the source of information rigidities in the expectations formation process," Journal of Economic Dynamics and Control, Elsevier, vol. 150(C).
    20. Alexandros Botsis & Christoph Görtz & Plutarchos Sakellaris, 2020. "Quantifying Qualitative Survey Data: New Insights on the (Ir)Rationality of Firms' Forecasts," CESifo Working Paper Series 8148, CESifo.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2305.02823. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.