IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2502.21206.html
   My bibliography  Save this paper

Chronologically Consistent Large Language Models

Author

Listed:
  • Songrun He
  • Linying Lv
  • Asaf Manela
  • Jimmy Wu

Abstract

Large language models are increasingly used in social sciences, but their training data can introduce lookahead bias and training leakage. A good chronologically consistent language model requires efficient use of training data to maintain accuracy despite time-restricted data. Here, we overcome this challenge by training a suite of chronologically consistent large language models, ChronoBERT and ChronoGPT, which incorporate only the text data that would have been available at each point in time. Despite this strict temporal constraint, our models achieve strong performance on natural language processing benchmarks, outperforming or matching widely used models (e.g., BERT), and remain competitive with larger open-weight models. Lookahead bias is model and application-specific because even if a chronologically consistent language model has poorer language comprehension, a regression or prediction model applied on top of the language model can compensate. In an asset pricing application predicting next-day stock returns from financial news, we find that ChronoBERT's real-time outputs achieve a Sharpe ratio comparable to state-of-the-art models, indicating that lookahead bias is modest. Our results demonstrate a scalable, practical framework to mitigate training leakage, ensuring more credible backtests and predictions across finance and other social science domains.

Suggested Citation

  • Songrun He & Linying Lv & Asaf Manela & Jimmy Wu, 2025. "Chronologically Consistent Large Language Models," Papers 2502.21206, arXiv.org, revised Mar 2025.
  • Handle: RePEc:arx:papers:2502.21206
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2502.21206
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Paul C. Tetlock & Maytal Saar‐Tsechansky & Sofus Macskassy, 2008. "More Than Words: Quantifying Language to Measure Firms' Fundamentals," Journal of Finance, American Finance Association, vol. 63(3), pages 1437-1467, June.
    2. Ledoit, Oliver & Wolf, Michael, 2008. "Robust performance hypothesis testing with the Sharpe ratio," Journal of Empirical Finance, Elsevier, vol. 15(5), pages 850-859, December.
    3. Jens Ludwig & Sendhil Mullainathan & Ashesh Rambachan, 2024. "Large Language Models: An Applied Econometric Framework," Papers 2412.07031, arXiv.org, revised Jan 2025.
    4. Zheng Tracy Ke & Bryan T. Kelly & Dacheng Xiu, 2019. "Predicting Returns With Text Data," NBER Working Papers 26186, National Bureau of Economic Research, Inc.
    5. Paul C. Tetlock, 2007. "Giving Content to Investor Sentiment: The Role of Media in the Stock Market," Journal of Finance, American Finance Association, vol. 62(3), pages 1139-1168, June.
    6. Paul Glasserman & Caden Lin, 2023. "Assessing Look-Ahead Bias in Stock Return Predictions Generated By GPT Sentiment Analysis," Papers 2309.17322, arXiv.org.
    7. Fama, Eugene F, 1970. "Efficient Capital Markets: A Review of Theory and Empirical Work," Journal of Finance, American Finance Association, vol. 25(2), pages 383-417, May.
    8. Jiang, Hao & Li, Sophia Zhengzi & Wang, Hao, 2021. "Pervasive underreaction: Evidence from high-frequency data," Journal of Financial Economics, Elsevier, vol. 141(2), pages 573-599.
    9. Tim Loughran & Bill Mcdonald, 2011. "When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10‐Ks," Journal of Finance, American Finance Association, vol. 66(1), pages 35-65, February.
    10. Manela, Asaf & Moreira, Alan, 2017. "News implied volatility and disaster concerns," Journal of Financial Economics, Elsevier, vol. 123(1), pages 137-162.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jeon, Yoontae & McCurdy, Thomas H. & Zhao, Xiaofei, 2022. "News as sources of jumps in stock returns: Evidence from 21 million news articles for 9000 companies," Journal of Financial Economics, Elsevier, vol. 145(2), pages 1-17.
    2. Luiz Renato Lima & Lucas Lúcio Godeiro, 2023. "Equity‐premium prediction: Attention is all you need," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(1), pages 105-122, January.
    3. Zheng Tracy Ke & Bryan T. Kelly & Dacheng Xiu, 2019. "Predicting Returns With Text Data," NBER Working Papers 26186, National Bureau of Economic Research, Inc.
    4. Yuan, Kaibin & Liang, Yuheng & Zhu, Mengnan, 2024. "Social forecasting: Online social opinion and the cross-section of stock returns," Pacific-Basin Finance Journal, Elsevier, vol. 86(C).
    5. Schnaubelt, Matthias & Fischer, Thomas G. & Krauss, Christopher, 2020. "Separating the signal from the noise – Financial machine learning for Twitter," Journal of Economic Dynamics and Control, Elsevier, vol. 114(C).
    6. Jesús Villota, 2025. "Predicting Market Reactions to News: An LLM-Based Approach Using Spanish Business Articles," Working Papers wp2025_2501, CEMFI.
    7. Alejandro Lopez-Lira & Yuehua Tang, 2023. "Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models," Papers 2304.07619, arXiv.org, revised Sep 2024.
    8. Sun, Andrew & Lachanski, Michael & Fabozzi, Frank J., 2016. "Trade the tweet: Social media text mining and sparse matrix factorization for stock market prediction," International Review of Financial Analysis, Elsevier, vol. 48(C), pages 272-281.
    9. García, Diego & Hu, Xiaowen & Rohrer, Maximilian, 2023. "The colour of finance words," Journal of Financial Economics, Elsevier, vol. 147(3), pages 525-549.
    10. An, Suwei, 2023. "Essays on incentive contracts, M&As, and firm risk," Other publications TiSEM dd97d2f5-1c9d-47c5-ba62-f, Tilburg University, School of Economics and Management.
    11. Gabriele Ranco & Ilaria Bordino & Giacomo Bormetti & Guido Caldarelli & Fabrizio Lillo & Michele Treccani, 2014. "Coupling news sentiment with web browsing data improves prediction of intra-day price dynamics," Papers 1412.3948, arXiv.org, revised Dec 2015.
    12. Su, Zhi & Lu, Man & Yin, Libo, 2018. "Oil prices and news-based uncertainty: Novel evidence," Energy Economics, Elsevier, vol. 72(C), pages 331-340.
    13. Goodell, John W. & Kumar, Satish & Lim, Weng Marc & Pattnaik, Debidutta, 2021. "Artificial intelligence and machine learning in finance: Identifying foundations, themes, and research clusters from bibliometric analysis," Journal of Behavioral and Experimental Finance, Elsevier, vol. 32(C).
    14. Mao, Huina & Counts, Scott & Bollen, Johan, 2015. "Quantifying the effects of online bullishness on international financial markets," Statistics Paper Series 09, European Central Bank.
    15. Aysan, Ahmet Faruk & Caporin, Massimiliano & Cepni, Oguzhan, 2024. "Not all words are equal: Sentiment and jumps in the cryptocurrency market," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 91(C).
    16. Loughran, Tim & McDonald, Bill & Pragidis, Ioannis, 2019. "Assimilation of oil news into prices," International Review of Financial Analysis, Elsevier, vol. 63(C), pages 105-118.
    17. Marie Bessec & Julien Fouquau, 2024. "A Green Wave in Media: A Change of Tack in Stock Markets," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 86(5), pages 1026-1057, October.
    18. Brière, Marie & Huynh, Karen & Laudy, Olav & Pouget, Sébastien, 2023. "Stock market reaction to news: Do tense and horizon matter?," Finance Research Letters, Elsevier, vol. 58(PD).
    19. Duan, Jiaxin & Kou, Fangyuan & Wang, Zining & Wei, Yixin, 2024. "When echoes surpass voices: Market reaction to forwarded news," International Review of Financial Analysis, Elsevier, vol. 96(PA).
    20. Prajwal Eachempati & Praveen Ranjan Srivastava, 2021. "Accounting for unadjusted news sentiment for asset pricing," Qualitative Research in Financial Markets, Emerald Group Publishing Limited, vol. 13(3), pages 383-422, May.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2502.21206. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.