IDEAS home Printed from https://ideas.repec.org/a/eee/jimfin/v157y2025ics0261560625001044.html

Economic uncertainty measures, experts and large language models

Author

Listed:
  • Bontempi, Maria Elena
  • Charemza, Wojciech
  • Makarova, Svetlana

Abstract

The paper proposes a randomness-type test for comparing the validity of different measures of economic uncertainty. The test verifies the randomness hypothesis for the match between the jumps of an uncertainty index and the dates of uncertainty-generating events named by the panel of experts or artificial intelligence through large language models (LLMs) capable of generating human-like text. The test can also be applied to verify whether LLMs provide a reliable selection of uncertainty-generating events. It was initially used to evaluate the quality of three uncertainty indices for Poland and then applied to six uncertainty indices for the US using monthly data from January 2004 to March 2021 for both countries. The results show that LLMs provide a reasonable alternative for testing when panels of experts are not available.

Suggested Citation

  • Bontempi, Maria Elena & Charemza, Wojciech & Makarova, Svetlana, 2025. "Economic uncertainty measures, experts and large language models," Journal of International Money and Finance, Elsevier, vol. 157(C).
    • Handle: RePEc:eee:jimfin:v:157:y:2025:i:c:s0261560625001044
    DOI: 10.1016/j.jimonfin.2025.103369
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0261560625001044
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jimonfin.2025.103369?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Huang, Yun & Luk, Paul, 2020. "Measuring economic policy uncertainty in China," China Economic Review, Elsevier, vol. 59(C).
    2. Stanislaw Bartha & Maria Elena Bontempi, 2022. "Measuring Economic Uncertainty for Poland," Working Papers wp1178, Dipartimento Scienze Economiche, Universita' di Bologna.
    3. Denton, Frank T, 1985. "Data Mining as an Industry," The Review of Economics and Statistics, MIT Press, vol. 67(1), pages 124-127, February.
    4. Al-Thaqeb, Saud Asaad & Algharabali, Barrak Ghanim, 2019. "Economic policy uncertainty: A literature review," The Journal of Economic Asymmetries, Elsevier, vol. 20(C).
    5. Efrem Castelnuovo & Guay Lim & Giovanni Pellegrino, 2017. "A Short Review of the Recent Literature on Uncertainty," Australian Economic Review, The University of Melbourne, Melbourne Institute of Applied Economic and Social Research, vol. 50(1), pages 68-78, March.
    6. Maria Elena Bontempi & Michele Frigeri & Roberto Golinelli & Matteo Squadrani, 2021. "EURQ: A New Web Search‐based Uncertainty Index," Economica, London School of Economics and Political Science, vol. 88(352), pages 969-1015, October.
    7. Danilo Cascaldi-Garcia & Cisil Sarisoy & Juan M. Londono & Bo Sun & Deepa D. Datta & Thiago Ferreira & Olesya Grishchenko & Mohammad R. Jahan-Parvar & Francesca Loria & Sai Ma & Marius Rodriguez & Ilk, 2023. "What Is Certain about Uncertainty?," Journal of Economic Literature, American Economic Association, vol. 61(2), pages 624-654, June.
    8. Kyle Jurado & Sydney C. Ludvigson & Serena Ng, 2015. "Measuring Uncertainty," American Economic Review, American Economic Association, vol. 105(3), pages 1177-1216, March.
    9. Wang, Li, 2022. "New testing procedures with k-FWER control for discrete data," Statistics & Probability Letters, Elsevier, vol. 180(C).
    10. Shinohara, Takeshi & Okuda, Tatsushi & Nakajima, Jouchi, 2021. "Characteristics of Uncertainty Indices in the Macroeconomy," Economic Review, Hitotsubashi University, vol. 72(3), pages 246-267, July.
    11. Ozturk, Ezgi O. & Sheng, Xuguang Simon, 2018. "Measuring global and country-specific uncertainty," Journal of International Money and Finance, Elsevier, vol. 88(C), pages 276-295.
    12. Andrew Patton & Dimitris Politis & Halbert White, 2009. "Correction to “Automatic Block-Length Selection for the Dependent Bootstrap” by D. Politis and H. White," Econometric Reviews, Taylor & Francis Journals, vol. 28(4), pages 372-375.
    13. Dai, Peng-Fei & Xiong, Xiong & Zhou, Wei-Xing, 2021. "A global economic policy uncertainty index from principal component analysis," Finance Research Letters, Elsevier, vol. 40(C).
    14. Shields, Kalvinder & Tran, Trung Duc, 2023. "Better understanding how uncertainty impacts the economy: Insights from internet search data on the importance of disaggregation," Macroeconomic Dynamics, Cambridge University Press, vol. 27(5), pages 1319-1344, July.
    15. Scott R. Baker & Nicholas Bloom & Steven J. Davis, 2016. "Measuring Economic Policy Uncertainty," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 131(4), pages 1593-1636.
    16. Sydney C. Ludvigson & Sai Ma & Serena Ng, 2021. "Uncertainty and Business Cycles: Exogenous Impulse or Endogenous Response?," American Economic Journal: Macroeconomics, American Economic Association, vol. 13(4), pages 369-410, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Carlos Cañizares Martínez & Arne Gieseck, 2025. "The effects of macro uncertainty shocks in the euro area: a FAVAR approach," Empirical Economics, Springer, vol. 68(6), pages 2829-2872, June.
    2. Michael Ryan, 2020. "An Anchor in Stormy Seas: Does Reforming Economic Institutions Reduce Uncertainty? Evidence from New Zealand," Working Papers in Economics 20/11, University of Waikato.
    3. Haining Chen & Prince Asare Vitenu-Sackey & Isaac Akpemah Bathuure, 2024. "Uncertainty Measures and Business Cycles: Evidence From the US," SAGE Open, , vol. 14(2), pages 21582440241, April.
    4. Saygin Sahinoz & Evren Erdogan Cosar, 2020. "Quantifying uncertainty and identifying its impacts on the Turkish economy," Empirica, Springer;Austrian Institute for Economic Research;Austrian Economic Association, vol. 47(2), pages 365-387, May.
    5. Giraldo, Carlos & Giraldo, Iader & Gomez-Gonzalez, Jose E. & Uribe, Jorge M., 2025. "US uncertainty shocks, credit, production, and prices: The case of fourteen Latin American countries," Research in International Business and Finance, Elsevier, vol. 78(C).
    6. Yujia, Li & Zixiang, Zhu & Ming, Che, 2024. "Exploring the relationship between China's economic policy uncertainty and business cycles: Exogenous impulse or endogenous responses?," Emerging Markets Review, Elsevier, vol. 58(C).
    7. Josué Diwambuena & Jean-Paul K. Tsasa, 2021. "The Real Effects of Uncertainty Shocks: New Evidence from Linear and Nonlinear SVAR Models," BEMPS - Bozen Economics & Management Paper Series BEMPS87, Faculty of Economics and Management at the Free University of Bozen.
    8. Beckmann, Joscha & Davidson, Sharada Nia & Koop, Gary & Schüssler, Rainer, 2023. "Cross-country uncertainty spillovers: Evidence from international survey data," Journal of International Money and Finance, Elsevier, vol. 130(C).
    9. Martin Iseringhausen & Konstantinos Theodoridis, 2025. "A survey-based measure of asymmetric macroeconomic risk in the euro area," Working Papers 68, European Stability Mechanism, revised 11 Feb 2025.
    10. Alina Bobasu & Lucia Quaglietti & Martino Ricci, 2024. "Tracking Global Economic Uncertainty: Implications for the Euro Area," IMF Economic Review, Palgrave Macmillan;International Monetary Fund, vol. 72(2), pages 820-857, June.
    11. Shen, Yifan & He, Jia & Shi, Xunpeng & Zeng, Ting, 2025. "Uncertainty, macroeconomic activity and commodity price: A global analysis," International Review of Financial Analysis, Elsevier, vol. 101(C).
    12. Costantini, Mauro & Sousa, Ricardo M., 2022. "What uncertainty does to euro area sovereign bond markets: Flight to safety and flight to quality," Journal of International Money and Finance, Elsevier, vol. 122(C).
    13. Jiang, Fuwei & Kang, Jie & Meng, Lingchao, 2024. "Certainty of uncertainty for asset pricing," Journal of Empirical Finance, Elsevier, vol. 78(C).
    14. Dibiasi, Andreas & Sarferaz, Samad, 2023. "Measuring macroeconomic uncertainty: A cross-country analysis," European Economic Review, Elsevier, vol. 153(C).
    15. Yi‐Ting Peng & Tsangyao Chang & Omid Ranjbar, 2022. "Analyzing the degree of persistence of economic policy uncertainty using linear and non‐linear fourier quantile unit root tests," Manchester School, University of Manchester, vol. 90(4), pages 453-471, July.
    16. Oscar Claveria, 2021. "On the Aggregation of Survey-Based Economic Uncertainty Indicators Between Different Agents and Across Variables," Journal of Business Cycle Research, Springer;Centre for International Research on Economic Tendency Surveys (CIRET), vol. 17(1), pages 1-26, April.
    17. Juan M. Londono & Sai Ma & Beth Anne Wilson, 2021. "The Global Transmission of Real Economic Uncertainty," International Finance Discussion Papers 1317, Board of Governors of the Federal Reserve System (U.S.).
    18. Che, Ming & Zhu, Zixiang & Li, Yujia, 2023. "Geopolitical risk and economic policy uncertainty: Different roles in China's financial cycle," International Review of Financial Analysis, Elsevier, vol. 90(C).
    19. Keijsers, Bart & van Dijk, Dick, 2025. "Does economic uncertainty predict real activity in real time?," International Journal of Forecasting, Elsevier, vol. 41(2), pages 748-762.
    20. Ferrer Fernández, María & Henry, Ólan & Pybis, Sam & Stamatogiannis, Michalis P., 2023. "Can we forecast better in periods of low uncertainty? The role of technical indicators," Journal of Empirical Finance, Elsevier, vol. 71(C), pages 1-12.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • C8 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs
    • C32 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes; State Space Models
    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness
    • E32 - Macroeconomics and Monetary Economics - - Prices, Business Fluctuations, and Cycles - - - Business Fluctuations; Cycles
    • E60 - Macroeconomics and Monetary Economics - - Macroeconomic Policy, Macroeconomic Aspects of Public Finance, and General Outlook - - - General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jimfin:v:157:y:2025:i:c:s0261560625001044. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/inca/30443 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.