IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2509.01110.html
   My bibliography  Save this paper

NoLBERT: A No Lookahead(back) Foundational Language Model for Empirical Research

Author

Listed:
  • Ali Kakhbod
  • Peiyao Li

Abstract

We present NoLBERT, a lightweight, timestamped foundational language model for empirical research in social sciences, particularly in economics and finance. By pre-training exclusively on 1976-1995 text, NoLBERT avoids both lookback and lookahead biases that can undermine econometric inference. It exceeds domain-specific baselines on NLP benchmarks while maintaining temporal consistency. Applied to patent texts, NoLBERT enables the construction of firm-level innovation networks and shows that gains in innovation centrality predict higher long-run profit growth.

Suggested Citation

  • Ali Kakhbod & Peiyao Li, 2025. "NoLBERT: A No Lookahead(back) Foundational Language Model for Empirical Research," Papers 2509.01110, arXiv.org.
    • Handle: RePEc:arx:papers:2509.01110
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2509.01110
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Leonid Kogan & Dimitris Papanikolaou & Amit Seru & Noah Stoffman, 2017. "Technological Innovation, Resource Allocation, and Growth," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 132(2), pages 665-712.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hickfang, Michael & Holder, Ulrike, 2018. "The impact of stock options on risk-taking: Founder-CEOs and innovation," Discussion Papers of the Institute for Organisational Economics 12/2018, University of Münster, Institute for Organisational Economics.
    2. Ilona Babenko & Benjamin Bennett & John M Bizjak & Jeffrey L Coles & Jason J Sandvik, 2023. "Clawback Provisions and Firm Risk," The Review of Corporate Finance Studies, Society for Financial Studies, vol. 12(2), pages 191-239.
    3. Zhao, Jun & Shahbaz, Muhammad & Dong, Kangyin, 2022. "How does energy poverty eradication promote green growth in China? The role of technological innovation," Technological Forecasting and Social Change, Elsevier, vol. 175(C).
    4. Kimura, Yosuke, 2024. "Market-based patent value of green transformation technologies," Finance Research Letters, Elsevier, vol. 68(C).
    5. Pauly, Stefan & Stipanicic, Fernando, 2021. "The creation and diffusion of knowledge: Evidence from the Jet Age," CEPREMAP Working Papers (Docweb) 2112, CEPREMAP.
    6. Simon Kinyua Njeru & Dr. Robert Mang’ana & Dr. Enos Anene, 2025. "Strategy Implementation and Performance of Manufacturing Pharmaceutical Companies in Kenya," Journal of Business and Strategic Management, CARI Journals Limited, vol. 10(5), pages 36-62.
    7. Stephen G. Dimmock & Jiekun Huang & Scott J. Weisbenner, 2022. "Give Me Your Tired, Your Poor, Your High-Skilled Labor: H-1B Lottery Outcomes and Entrepreneurial Success," Management Science, INFORMS, vol. 68(9), pages 6950-6970, September.
    8. Alex Bell & Raj Chetty & Xavier Jaravel & Neviana Petkova & John Van Reenen, 2019. "Who Becomes an Inventor in America? The Importance of Exposure to Innovation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 134(2), pages 647-713.
    9. Zhou, Xiaoxiao & Jia, Mengyu & Li, Wenqing & Zhao, Xin & Gatto, Andrea & Ma, Xiaowei, 2024. "Higher education or scientific research: Which one contributes more to China's green innovation?," Socio-Economic Planning Sciences, Elsevier, vol. 94(C).
    10. Kim, Jinhee & Lee, Keun, 2022. "Local–global interface as a key factor in the catching up of regional innovation systems: Fast versus slow catching up among Taipei, Shenzhen, and Penang in Asia," Technological Forecasting and Social Change, Elsevier, vol. 174(C).
    11. Huasheng Song & Chao Zhang, 2024. "Land regulations, innovation and productivity: Firm‐level evidence from China," The World Economy, Wiley Blackwell, vol. 47(4), pages 1387-1426, April.
    12. Yusuke Oh & Koji Takahashi, 2020. "R&D and Innovation: Evidence from Patent Data," Bank of Japan Working Paper Series 20-E-7, Bank of Japan.
    13. Danilo Cascaldi-Garcia & Marija Vukotic, 2022. "Patent-Based News Shocks," The Review of Economics and Statistics, MIT Press, vol. 104(1), pages 51-66, March.
    14. Jungbae Kim, 2024. "The effect of PCAOB inspections on corporate innovation: evidence from deficiencies about the valuation of intangibles," Review of Accounting Studies, Springer, vol. 29(2), pages 1491-1523, June.
    15. Gu, Yuqi & Zhang, Ling, 2017. "The impact of the Sarbanes-Oxley Act on corporate innovation," Journal of Economics and Business, Elsevier, vol. 90(C), pages 17-30.
    16. Hötte, Kerstin & Pichler, Anton & Lafond, François, 2021. "The rise of science in low-carbon energy technologies," Renewable and Sustainable Energy Reviews, Elsevier, vol. 139(C).
    17. Cozzi, Guido & Pataracchia, Beatrice & Pfeiffer, Philipp & Marco, Ratto, 2017. "How much Keynes and how much Schumpeter? An Estimated Macromodel of the US Economy," JRC Working Papers in Economics and Finance 2017-01, Joint Research Centre, European Commission.
    18. Hasan, Iftekhar & Li, Xiang & Takalo, Tuomas, 2023. "Technological innovation and the bank lending channel of monetary policy transmission," IWH Discussion Papers 14/2021, Halle Institute for Economic Research (IWH), revised 2023.
    19. Mammadaliyev, Farid & Gilsing, Victor & Knoben, J., 2024. "How do firms adapt their portfolios of external collaborations to changing internal organizational attributes? The moderating role of firm age," Other publications TiSEM 56c854b7-ed86-4830-843e-c, Tilburg University, School of Economics and Management.
    20. Pu Liu & Yingying Shao, 2022. "Innovation and new business formation: the role of innovative large firms," Small Business Economics, Springer, vol. 59(2), pages 691-720, August.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2509.01110. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.