The Importance of Data Quality in Sentiment Analysis: A Comparative Study of VADER and RoBERTa

In the era of digitalization, data can be considered as an asset with significant importance. Its quality is crucial, especially when conducting various types of analyses that examine trends. Even minor inaccuracies in the input stream can lead to deviations in results and subsequent misinterpretations. This study explores the impact of noisy data on the effectiveness of sentiment analysis models. A use case was conducted, demonstrating two different approaches: one is a lexicon-based method Valence Aware Dictionary and Sentiment Reasoner, while the other algorithm is based on deep neural networks and is trained on large datasets called Robustly Optimized Bidirectional Encoder Representations from Transformers Pretraining Approach. The analysis is performed over a dataset of user opinions from the social media platform Twitter. Additionally, the potential for combining the outcomes of both models to achieve a more comprehensive and reliable interpretation is assessed. The study highlights the need of data preprocessing and quality control measures to ensure higher accuracy and reliability in sentiment analysis. A conceptual model and a prototype module for improving data quality are proposed, designed to assist in the initial processing of input data. This research provides guidelines for enhancing the productivity of sentiment analysis in social media and lays the foundation for future studies focused on optimizing natural language process models.