IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2512.23847.html

A Test of Lookahead Bias in LLM Forecasts

Author

Listed:
  • Zhenyu Gao
  • Wenxi Jiang
  • Yutong Yan

Abstract

We develop a statistical test to detect lookahead bias in economic forecasts generated by large language models (LLMs). Using state-of-the-art pre-training data detection techniques, we estimate the likelihood that a given prompt appeared in an LLM's training corpus, a statistic we term Lookahead Propensity (LAP). We formally show that a positive correlation between LAP and forecast accuracy indicates the presence and magnitude of lookahead bias, and apply the test to two forecasting tasks: news headlines predicting stock returns and earnings call transcripts predicting capital expenditures. Our test provides a cost-efficient, diagnostic tool for assessing the validity and reliability of LLM-generated forecasts.

Suggested Citation

  • Zhenyu Gao & Wenxi Jiang & Yutong Yan, 2025. "A Test of Lookahead Bias in LLM Forecasts," Papers 2512.23847, arXiv.org.
    • Handle: RePEc:arx:papers:2512.23847
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2512.23847
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Sean Cao Robert H. Smith & Wei Jiang & Baozhong Yang J. Mack Robinson & Alan L Zhang & Tarun Ramadorai, 2023. "How to Talk When a Machine Is Listening: Corporate Disclosure in the Age of AI," The Review of Financial Studies, Society for Financial Studies, vol. 36(9), pages 3603-3642.
    2. Alejandro Lopez-Lira & Yuehua Tang & Mingyin Zhu, 2025. "The Memorization Problem: Can We Trust LLMs' Economic Forecasts?," Papers 2504.14765, arXiv.org, revised Dec 2025.
    3. Malcolm Baker & Jeffrey Wurgler, 2006. "Investor Sentiment and the Cross‐Section of Stock Returns," Journal of Finance, American Finance Association, vol. 61(4), pages 1645-1680, August.
    4. Songrun He & Linying Lv & Asaf Manela & Jimmy Wu, 2025. "Chronologically Consistent Large Language Models," Papers 2502.21206, arXiv.org, revised Jul 2025.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yutong Yan & Raphael Tang & Zhenyu Gao & Wenxi Jiang & Yao Lu, 2026. "DatedGPT: Preventing Lookahead Bias in Large Language Models with Time-Aware Pretraining," Papers 2603.11838, arXiv.org.
    2. Yaxuan Kong & Hoyoung Lee & Yoontae Hwang & Alejandro Lopez-Lira & Bradford Levy & Dhagash Mehta & Qingsong Wen & Chanyeol Choi & Yongjae Lee & Stefan Zohren, 2026. "Evaluating LLMs in Finance Requires Explicit Bias Consideration," Papers 2602.14233, arXiv.org.
    3. Mostapha Benhenda, 2026. "Look-Ahead-Bench: a Standardized Benchmark of Look-ahead Bias in Point-in-Time LLMs for Finance," Papers 2601.13770, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Alexander Eliseev & Sergei Seleznev, 2026. "Fake Date Tests: Can We Trust In-sample Accuracy of LLMs in Macroeconomic Forecasting?," Papers 2601.07992, arXiv.org, revised Mar 2026.
    2. Mostapha Benhenda, 2026. "Look-Ahead-Bench: a Standardized Benchmark of Look-ahead Bias in Point-in-Time LLMs for Finance," Papers 2601.13770, arXiv.org.
    3. Didisheim, Antoine & Fraschini, Martina & Somoza, Luciano, 2025. "AI’s predictable memory in financial analysis," Economics Letters, Elsevier, vol. 256(C).
    4. Ke Wu & Baozhong Yang & Zhenkun Ying & Dexin Zhou, 2025. "Anonymization and Information Loss," Papers 2511.15364, arXiv.org.
    5. Ding, Rui & Guo, Jintong & Zhang, Min, 2024. "Practice a poker face: Manager emotion and investor sentiment," Pacific-Basin Finance Journal, Elsevier, vol. 85(C).
    6. David Hirshleifer & Danling Jiang, 2010. "A Financing-Based Misvaluation Factor and the Cross-Section of Expected Returns," The Review of Financial Studies, Society for Financial Studies, vol. 23(9), pages 3401-3436.
    7. Lepori, Gabriele M., 2015. "Investor mood and demand for stocks: Evidence from popular TV series finales," Journal of Economic Psychology, Elsevier, vol. 48(C), pages 33-47.
    8. Muhammad Kashif & Thomas Leirvik, 2022. "The MAX Effect in an Oil Exporting Country: The Case of Norway," JRFM, MDPI, vol. 15(4), pages 1-16, March.
    9. Caginalp, Gunduz & DeSantis, Mark, 2017. "Does price efficiency increase with trading volume? Evidence of nonlinearity and power laws in ETFs," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 467(C), pages 436-452.
    10. Dash, Saumya Ranjan & Maitra, Debasish, 2018. "Does sentiment matter for stock returns? Evidence from Indian stock market using wavelet approach," Finance Research Letters, Elsevier, vol. 26(C), pages 32-39.
    11. Klaus Grobys & James W. Kolari & Jere Rutanen, 2022. "Factor momentum, option-implied volatility scaling, and investor sentiment," Journal of Asset Management, Palgrave Macmillan, vol. 23(2), pages 138-155, March.
    12. Constantinos Antoniou & John A. Doukas & Avanidhar Subrahmanyam, 2016. "Investor Sentiment, Beta, and the Cost of Equity Capital," Management Science, INFORMS, vol. 62(2), pages 347-367, February.
    13. Papapostolou, Nikos C. & Pouliasis, Panos K. & Nomikos, Nikos K. & Kyriakou, Ioannis, 2016. "Shipping investor sentiment and international stock return predictability," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 96(C), pages 81-94.
    14. Chen, Catherine Huirong & Choy, Siu Kai & Tan, Yongxian, 2022. "The cash conversion cycle spread: International evidence," Journal of Banking & Finance, Elsevier, vol. 140(C).
    15. David E. Allen & Michael McAleer & Abhay K. Singh, 2019. "Daily market news sentiment and stock prices," Applied Economics, Taylor & Francis Journals, vol. 51(30), pages 3212-3235, June.
    16. Ben-Rephael, Azi & Kandel, Shmuel & Wohl, Avi, 2012. "Measuring investor sentiment with mutual fund flows," Journal of Financial Economics, Elsevier, vol. 104(2), pages 363-382.
    17. Yousaf, Imran & Youssef, Manel & Goodell, John W., 2022. "Quantile connectedness between sentiment and financial markets: Evidence from the S&P 500 twitter sentiment index," International Review of Financial Analysis, Elsevier, vol. 83(C).
    18. Shahzad, Syed Jawad Hussain & Raza, Naveed & Balcilar, Mehmet & Ali, Sajid & Shahbaz, Muhammad, 2017. "Can economic policy uncertainty and investors sentiment predict commodities returns and volatility?," Resources Policy, Elsevier, vol. 53(C), pages 208-218.
    19. Cakici, Nusret & Zaremba, Adam, 2022. "Salience theory and the cross-section of stock returns: International and further evidence," Journal of Financial Economics, Elsevier, vol. 146(2), pages 689-725.
    20. Lanfeng Kao & Min-Hsien Chiang & Anlin Chen, 2024. "To mitigate the effect of underwriter bargaining power on IPO pricing through audit committee under an economy of little information asymmetry," Review of Quantitative Finance and Accounting, Springer, vol. 63(1), pages 147-167, July.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2512.23847. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.