Beyond news headlines and TF-IDF: Enhancing text-based forecasting models with validated collocations and improved attention

This paper proposes a method to improve text-based forecasting models, specifically for crude oil prices. Utilizing advanced techniques, including pattern validation and attention mechanisms, the study demonstrates notable improvements in predictive power over traditional approaches. A key finding is that considering the full text of news articles, rather than limiting the analysis to headlines, yields significant gains in forecasting accuracy. Furthermore, the model featuring verb-noun and noun-verb collocation pattern validation consistently outperforms benchmarks and models based solely on news headlines across various forecasting horizons. The results suggest that the presence of collocations such as ‘price fell’, ‘prices tumbled’, and ‘price dropped’ in crude-oil-related news articles is associated with lower oil price returns. Additionally, integrating macroeconomic data with text-based features enhances predictive performance, demonstrating that combining structured economic indicators with textual features improves forecasting accuracy.