IDEAS home Printed from https://ideas.repec.org/a/eee/ecolet/v256y2025ics0165176525004598.html

Measuring economic sentiment from open-ended survey comments using large language models

Author

Listed:
  • Seiler, Pascal

Abstract

This article develops a novel economic sentiment indicator (LLM-ESI) by applying large language models to open-ended responses from Swiss business tendency surveys. Using a BERT-based transformer model, it extracts firm-level sentiment from free-text survey comments and aggregates it into a high-frequency indicator of macroeconomic conditions. The LLM-ESI closely tracks the business cycle and performs on par with, or better than, traditional benchmarks in nowcasting GDP. These results highlight the potential of large language models and open-ended survey responses to deliver timely and nuanced signals for real-time economic analysis.

Suggested Citation

  • Seiler, Pascal, 2025. "Measuring economic sentiment from open-ended survey comments using large language models," Economics Letters, Elsevier, vol. 256(C).
    • Handle: RePEc:eee:ecolet:v:256:y:2025:i:c:s0165176525004598
    DOI: 10.1016/j.econlet.2025.112622
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0165176525004598
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.econlet.2025.112622?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-1084, September.
    2. Beatrice Ferrario & Stefanie Stantcheva, 2022. "Eliciting People's First-Order Concerns: Text Analysis of Open-Ended Survey Questions," AEA Papers and Proceedings, American Economic Association, vol. 112, pages 163-169, May.
    3. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    4. Scott R. Baker & Nicholas Bloom & Steven J. Davis, 2016. "Measuring Economic Policy Uncertainty," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 131(4), pages 1593-1636.
    5. Altig, Dave & Baker, Scott & Barrero, Jose Maria & Bloom, Nicholas & Bunn, Philip & Chen, Scarlet & Davis, Steven J. & Leather, Julia & Meyer, Brent & Mihaylov, Emil & Mizen, Paul & Parker, Nicholas &, 2020. "Economic uncertainty before and during the COVID-19 pandemic," Journal of Public Economics, Elsevier, vol. 191(C).
    6. Yuriy Gorodnichenko & Tho Pham & Oleksandr Talavera, 2023. "The Voice of Monetary Policy," American Economic Review, American Economic Association, vol. 113(2), pages 548-584, February.
    7. Marc Burri & Daniel Kaufmann, 2020. "A daily fever curve for the Swiss economy," Swiss Journal of Economics and Statistics, Springer;Swiss Society of Economics and Statistics, vol. 156(1), pages 1-11, December.
    8. Miguel Faria-e-Castro & Fernando Leibovici, 2024. "Artificial Intelligence and Inflation Forecasts," Review, Federal Reserve Bank of St. Louis, vol. 106(12), pages 1-14, November.
    9. Shapiro, Adam Hale & Sudhof, Moritz & Wilson, Daniel J., 2022. "Measuring news sentiment," Journal of Econometrics, Elsevier, vol. 228(2), pages 221-243.
    10. Philipp Wegmueller & Christian Glocker, 2024. "Capturing Swiss economic confidence," Swiss Journal of Economics and Statistics, Springer;Swiss Society of Economics and Statistics, vol. 160(1), pages 1-17, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Vegard H. Larsen & Leif Anders Thorsrud, 2026. "Using Transformers and Reinforcement Learning as Narrative Filters in Macroeconomics," CESifo Working Paper Series 12454, CESifo.
    2. Aprigliano, Valentina & Emiliozzi, Simone & Guaitoli, Gabriele & Luciani, Andrea & Marcucci, Juri & Monteforte, Libero, 2023. "The power of text-based indicators in forecasting Italian economic activity," International Journal of Forecasting, Elsevier, vol. 39(2), pages 791-808.
    3. Miescu, Mirela & Rossi, Raffaele, 2021. "COVID-19-induced shocks and uncertainty," European Economic Review, Elsevier, vol. 139(C).
    4. Marc Burri & Daniel Kaufmann, 2020. "A daily fever curve for the Swiss economy," Swiss Journal of Economics and Statistics, Springer;Swiss Society of Economics and Statistics, vol. 156(1), pages 1-11, December.
    5. Granziera, Eleonora & Sekhposyan, Tatevik, 2019. "Predicting relative forecasting performance: An empirical investigation," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1636-1657.
    6. Ma, Feng & Liu, Jing & Wahab, M.I.M. & Zhang, Yaojie, 2018. "Forecasting the aggregate oil price volatility in a data-rich environment," Economic Modelling, Elsevier, vol. 72(C), pages 320-332.
    7. Jon Ellingsen & Vegard H. Larsen & Leif Anders Thorsrud, 2022. "News media versus FRED‐MD for macroeconomic forecasting," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(1), pages 63-81, January.
    8. Xiafei Li & Yu Wei & Xiaodan Chen & Feng Ma & Chao Liang & Wang Chen, 2022. "Which uncertainty is powerful to forecast crude oil market volatility? New evidence," International Journal of Finance & Economics, John Wiley & Sons, Ltd., vol. 27(4), pages 4279-4297, October.
    9. Mariia Artemova & Francisco Blasques & Siem Jan Koopman & Zhaokun Zhang, 2021. "Forecasting in a changing world: from the great recession to the COVID-19 pandemic," Tinbergen Institute Discussion Papers 21-006/III, Tinbergen Institute.
    10. Ibrahim D. Raheem & Xuan Vinh Vo, 2022. "A new approach to exchange rate forecast: The role of global financial cycle and time‐varying parameters," International Journal of Finance & Economics, John Wiley & Sons, Ltd., vol. 27(3), pages 2836-2848, July.
    11. Saiz, Lorena & Ashwin, Julian & Kalamara, Eleni, 2021. "Nowcasting euro area GDP with news sentiment: a tale of two crises," Working Paper Series 2616, European Central Bank.
    12. Daniel Borup & Jonas N. Eriksen & Mads M. Kjær & Martin Thyrsgaard, 2024. "Predicting Bond Return Predictability," Management Science, INFORMS, vol. 70(2), pages 931-951, February.
    13. Gong, Yuting & Peng, Yongyuan & Xu, Luxuan & Chen, Kecai & Shi, Wenming, 2025. "Shipping news sentiment as a predictor of iron ore freight rates: Hybrid evidence from lexicon-based analysis and threshold autoregression modelling," Transport Policy, Elsevier, vol. 169(C), pages 178-190.
    14. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    15. Cao, Charles & Simin, Timothy & Xiao, Han, 2020. "Predicting the equity premium with the implied volatility spread," Journal of Financial Markets, Elsevier, vol. 51(C).
    16. Cao, Charles & Simin, Timothy & Xiao, Han, 2019. "Predicting the equity premium with the implied volatility spread," MPRA Paper 103651, University Library of Munich, Germany.
    17. Seo, Beomseok, 2025. "Econometric forecasting using ubiquitous news text: Text-enhanced factor model," International Journal of Forecasting, Elsevier, vol. 41(3), pages 1055-1072.
    18. Mei, Dexiang & Zeng, Qing & Zhang, Yaojie & Hou, Wenjing, 2018. "Does US Economic Policy Uncertainty matter for European stock markets volatility?," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 512(C), pages 215-221.
    19. Borup, Daniel & Rapach, David E. & Schütte, Erik Christian Montes, 2023. "Mixed-frequency machine learning: Nowcasting and backcasting weekly initial claims with daily internet search volume data," International Journal of Forecasting, Elsevier, vol. 39(3), pages 1122-1144.
    20. Simionescu, Mihaela, 2022. "Econometrics of sentiments- sentometrics and machine learning: The improvement of inflation predictions in Romania using sentiment analysis," Technological Forecasting and Social Change, Elsevier, vol. 182(C).

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • C55 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Large Data Sets: Modeling and Analysis
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • E32 - Macroeconomics and Monetary Economics - - Prices, Business Fluctuations, and Cycles - - - Business Fluctuations; Cycles
    • E37 - Macroeconomics and Monetary Economics - - Prices, Business Fluctuations, and Cycles - - - Forecasting and Simulation: Models and Applications
    • E66 - Macroeconomics and Monetary Economics - - Macroeconomic Policy, Macroeconomic Aspects of Public Finance, and General Outlook - - - General Outlook and Conditions

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ecolet:v:256:y:2025:i:c:s0165176525004598. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/ecolet .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.