AT-FinGPT: Financial risk prediction via an audio-text large language model

Financial risk prediction is crucial for investment decision-making. Traditional machine learning methods are limited by their structures and parameter size, which hinders their generalizability and effectiveness. Large language models (LLMs), which are pretrained with very large dataset and many GPUs have recently shown promising improvements in financial risk prediction. Despite this progress, most existing financial LLMs mainly rely on textual data for training and prediction, overlooking audio data and limiting analysis to text summarization. However, natural language processing studies have shown that audio from CEOs’ quarterly earnings calls is crucial for financial risk prediction. In this work, we introduce an audio–text LLM named AT-FinGPT, which fuses financial audio data and summarization texts for financial risk prediction. The empirical experimental results show that AT-FinGPT is superior to most advanced methods. Through an ablation study, we demonstrate that different data sources can facilitate financial risk assessment and discuss the effectiveness of each part in the AT-FinGPT model.