IDEAS home Printed from https://ideas.repec.org/p/pie/dsedps/2024-317.html
   My bibliography  Save this paper

Natural Language Processing Techniques for Long Financial Document

Author

Listed:
  • Maria Saveria Mavillonio

Abstract

In finance, Natural Language Processing (NLP) has become both a powerful and challenging tool, as extensive unstructured documents—such as business plans, financial reports, and regulatory filings—hold essential insights for strategic decision-making. This paper reviews the progression of NLP text representation methods, from foundational models to advanced Transformer architectures that greatly enhance semantic and contextual analysis. Yet, these models encounter limitations when applied to long financial documents, where computational efficiency and contextual coherence are critical. Recent innovations, including sparse attention mechanisms and domain-specific model adaptations, have improved the processing of lengthy texts, allowing for more accurate analysis of financial documents by capturing field-specific semantics. This paper also highlights the transformative role of NLP in financial analysis, especially where structured data is limited. Selecting the most suitable model for specific tasks is essential for maximizing NLP's impact in finance. Organized to provide a thorough overview, the paper covers text representation techniques, strategies for handling long texts, and applications in finance, establishing a foundation for advancing NLP-driven data analysis in this field.

Suggested Citation

  • Maria Saveria Mavillonio, 2024. "Natural Language Processing Techniques for Long Financial Document," Discussion Papers 2024/317, Dipartimento di Economia e Management (DEM), University of Pisa, Pisa, Italy.
    • Handle: RePEc:pie:dsedps:2024/317
    Note: ISSN 2039-1854
    as

    Download full text from publisher

    File URL: https://www.ec.unipi.it/documents/Ricerca/papers/2024-317.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Luca Barbaglia & Sergio Consoli & Sebastiano Manzan, 2023. "Forecasting with Economic News," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 41(3), pages 708-719, July.
    2. Elliott Ash & Stephen Hansen, 2023. "Text Algorithms in Economics," Annual Review of Economics, Annual Reviews, vol. 15(1), pages 659-688, September.
    3. Angelico, Cristina & Marcucci, Juri & Miccoli, Marcello & Quarta, Filippo, 2022. "Can we measure inflation expectations using Twitter?," Journal of Econometrics, Elsevier, vol. 228(2), pages 259-277.
    4. Brandyn Bok & Daniele Caratelli & Domenico Giannone & Argia M. Sbordone & Andrea Tambalotti, 2018. "Macroeconomic Nowcasting and Forecasting with Big Data," Annual Review of Economics, Annual Reviews, vol. 10(1), pages 615-643, August.
    5. Kathleen Weiss Hanley & Gerard Hoberg, 2019. "Dynamic Interpretation of Emerging Risks in the Financial Sector," The Review of Financial Studies, Society for Financial Studies, vol. 32(12), pages 4543-4603.
    6. Mark A Chen & Qinxi Wu & Baozhong Yang, 2019. "How Valuable Is FinTech Innovation?," The Review of Financial Studies, Society for Financial Studies, vol. 32(5), pages 2062-2106.
    7. Itay Goldstein & Chester S Spatt & Mao Ye, 2021. "Big Data in Finance," NBER Chapters, in: Big Data: Long-Term Implications for Financial Markets and Firms, pages 3213-3225, National Bureau of Economic Research, Inc.
    8. Kai Li & Feng Mai & Rui Shen & Xinyan Yan, 2021. "Measuring Corporate Culture Using Machine Learning," NBER Chapters, in: Big Data: Long-Term Implications for Financial Markets and Firms, pages 3265-3315, National Bureau of Economic Research, Inc.
    9. Gustaf Bellstam & Sanjai Bhagat & J. Anthony Cookson, 2021. "A Text-Based Analysis of Corporate Innovation," Management Science, INFORMS, vol. 67(7), pages 4004-4031, July.
    10. Kai Li & Feng Mai & Rui Shen & Xinyan Yan, 2021. "Measuring Corporate Culture Using Machine Learning [Machine learning methods that economists should know about]," The Review of Financial Studies, Society for Financial Studies, vol. 34(7), pages 3265-3315.
    11. Itay Goldstein & Chester S. Spatt & Mao Ye, 2021. "Big Data in Finance," NBER Working Papers 28615, National Bureau of Economic Research, Inc.
    12. Itay Goldstein & Chester S Spatt & Mao Ye, 2021. "Big Data in Finance [Institutional order handling and broker-affiliated trading venues]," The Review of Financial Studies, Society for Financial Studies, vol. 34(7), pages 3213-3225.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Michalski, Lachlan & Low, Rand Kwong Yew, 2024. "Determinants of corporate credit ratings: Does ESG matter?," International Review of Financial Analysis, Elsevier, vol. 94(C).
    2. Cao, Sean & Jiang, Wei & Wang, Junbo & Yang, Baozhong, 2024. "From Man vs. Machine to Man + Machine: The art and AI of stock analyses," Journal of Financial Economics, Elsevier, vol. 160(C).
    3. Li, Ang & Liu, Mark & Sheather, Simon, 2023. "Predicting stock splits using ensemble machine learning and SMOTE oversampling," Pacific-Basin Finance Journal, Elsevier, vol. 78(C).
    4. Yang, Jinglan & Liu, Jianghuai & Yao, Zheng & Ma, Chaoqun, 2024. "Measuring digitalization capabilities using machine learning," Research in International Business and Finance, Elsevier, vol. 70(PB).
    5. Rad, Hossein & Low, Rand Kwong Yew & Miffre, Joëlle & Faff, Robert, 2023. "The commodity risk premium and neural networks," Journal of Empirical Finance, Elsevier, vol. 74(C).
    6. Mahsa Samsami & Ralf Wagner, 2021. "Investment Decisions with Endogeneity: A Dirichlet Tree Analysis," JRFM, MDPI, vol. 14(7), pages 1-19, July.
    7. Niu, Yuhao & Wang, Sai & Wen, Wen & Li, Sifei, 2023. "Does digital transformation speed up dynamic capital structure adjustment? Evidence from China," Pacific-Basin Finance Journal, Elsevier, vol. 79(C).
    8. Wang, Sai & Wen, Wen & Niu, Yuhao & Li, Xin, 2024. "Digital transformation and corporate labor investment efficiency," Emerging Markets Review, Elsevier, vol. 59(C).
    9. Edmans, Alex & Fernandez-Perez, Adrian & Garel, Alexandre & Indriawan, Ivan, 2022. "Music sentiment and stock returns around the world," Journal of Financial Economics, Elsevier, vol. 145(2), pages 234-254.
    10. Day, Min-Yuh & Ni, Yensen, 2023. "Be greedy when others are fearful: Evidence from a two-decade assessment of the NDX 100 and S&P 500 indexes," International Review of Financial Analysis, Elsevier, vol. 90(C).
    11. Arnold, Lutz G. & Russ, David, 2024. "Listening to the noise: On price efficiency with dynamic trading," International Review of Economics & Finance, Elsevier, vol. 93(PB), pages 103-120.
    12. Wang, Yichen & Hu, Jun & Chen, Jia, 2023. "Does Fintech facilitate cross-border M&As? Evidence from Chinese A-share listed firms," International Review of Financial Analysis, Elsevier, vol. 85(C).
    13. Hałaj, Grzegorz & Martinez-Jaramillo, Serafin & Battiston, Stefano, 2024. "Financial stability through the lens of complex systems," Journal of Financial Stability, Elsevier, vol. 71(C).
    14. Zhang, Yaojie & Wahab, M.I.M. & Wang, Yudong, 2023. "Forecasting crude oil market volatility using variable selection and common factor," International Journal of Forecasting, Elsevier, vol. 39(1), pages 486-502.
    15. Yao Ge & Zheng Qiao & Zhe Shen & Zhiyu Zhang, 2023. "Production similarity and the cross‐section of stock returns: A machine learning approach," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 63(5), pages 4849-4882, December.
    16. Sun, Yue & Chai, Nana & Dong, Yizhe & Shi, Baofeng, 2022. "Assessing and predicting small industrial enterprises’ credit ratings: A fuzzy decision-making approach," International Journal of Forecasting, Elsevier, vol. 38(3), pages 1158-1172.
    17. Zijia Du & Alan Guoming Huang & Russ Wermers & Wenfeng Wu, 2022. "Language and Domain Specificity: A Chinese Financial Sentiment Dictionary [The effects of analyst-country institutions on biased research: Evidence from target prices]," Review of Finance, European Finance Association, vol. 26(3), pages 673-719.
    18. Hoang, Daniel & Wiegratz, Kevin, 2022. "Machine learning methods in finance: Recent applications and prospects," Working Paper Series in Economics 158, Karlsruhe Institute of Technology (KIT), Department of Economics and Management.
    19. Fengler, Matthias & Phan, Minh Tri, 2023. "A Topic Model for 10-K Management Disclosures," Economics Working Paper Series 2307, University of St. Gallen, School of Economics and Political Science.
    20. Victor Duarte & Diogo Duarte & Dejanir H. Silva, 2024. "Machine Learning for Continuous-Time Finance," CESifo Working Paper Series 10909, CESifo.

    More about this item

    Keywords

    Long Text; Financial Document Representation; Natural Language Processing; Transformers;
    All these keywords.

    JEL classification:

    • C45 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - Neural Networks and Related Topics
    • G2 - Financial Economics - - Financial Institutions and Services
    • G23 - Financial Economics - - Financial Institutions and Services - - - Non-bank Financial Institutions; Financial Instruments; Institutional Investors
    • L26 - Industrial Organization - - Firm Objectives, Organization, and Behavior - - - Entrepreneurship

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pie:dsedps:2024/317. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/dspisit.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.