More than a feeling. Introducing an NLP-based media sentiment index for the Berlin Stock Exchange, 1872–1930

Collective emotions or sentiment can exert significant influence on financial markets. This paper introduces newly curated data representing sentiment at the Berlin stock exchange from 1872 to 1930, utilizing daily market reports published in the Berliner Börsen-Zeitung. Employing an integrated workflow comprising optical character recognition, layout detection, machine learning, and natural language processing, we construct aspect-based sentiment data based on daily market observations. Although sentiment analysis may be considered a conventional task in certain domains, it presents several technical, conceptual, and historiographical challenges within the specific domain investigated in this study, namely historical financial news documents. To provide context for this project and to comprehend the somewhat elusive concept of market sentiment, we initially summarize relevant literature on the subject. Subsequently, we outline our workflow for extracting data from raw newspaper scans, emphasizing our efforts to ensure maximum data reliability. Notably, we illuminate key domain-specific challenges ranging from source-related issues to classification models. Finally, we conduct a comprehensive assessment of the data, addressing both technical and historiographical aspects, and propose potential future applications.