From Financial News To Price Forecasts: Assessing The Role Of Nlp-Based Sentiment Analysis And Variable Selection In Stock Price Prediction

This study investigates the impact of NLP-based sentiment analysis and variable selection techniques on stock price prediction. Sentiment indicators are extracted using Natural Language Processing (NLP) methods, including TextBlob-based sentiment polarity scores, VADER compound scores, and domainspecific lexicon-based sentiment scores. To address the complexity and dimensionality of financial data, variable selection techniques (PCA, LASSO, Elastic Net, PCA + LASSO, and PCA + Elastic Net) are employed. These methods help in constructing a more efficient feature set by reducing noise and multicollinearity. The selected features, combined with sentiment variables, are utilized in predictive models including ARIMAX, ANN, LSTM, and GRU. The models are tested on eight publicly traded stocks (AAPL, AMZN, GOOG, META, MSFT, NFLX, NVDA, TSLA) over a four-year period. The results indicate that the inclusion of sentiment variables improves forecasting performance, particularly when combined with dimensionality reduction and regularization techniques. Among these approaches, combinations of PCA with regularization techniques often lead to more stable and competitive forecasting performance. The findings highlight the value of integrating unstructured textual data from financial news into time series forecasting models, contributing to improved predictive performance in stock market applications.