Advancing tourism sentiment analysis: a comparative evaluation of traditional machine learning, deep learning, and transformer models on imbalanced datasets

Tourism sentiment analysis faces substantial challenges due to class imbalance and the complex linguistic features of user-generated content. This study systematically compares eight sentiment classification models, spanning traditional machine learning (naïve Bayes, support vector machines, logistic regression), deep learning (convolutional neural networks, long short-term memory networks [LSTMs], gated recurrent units [GRUs]), and transformer-based architectures (RoBERTa in two configurations: pretrained and fine-tuned), using a dataset of 505,980 TripAdvisor reviews. We evaluate model performance under imbalanced class conditions and examine the effectiveness of three oversampling techniques—SMOTE, ADASYN, and RandomOverSampler—in mitigating class bias. The results reveal significant performance disparities across architectures. Deep learning models, particularly LSTM (91.06% accuracy, Cohen’s kappa = 0.6846) and GRU (90.82% accuracy, Cohen’s kappa = 0.6781), consistently outperform traditional approaches. Fine-tuned RoBERTa achieved the highest performance, with 92.31% accuracy, a 95.34% F1-score, and Cohen’s kappa = 0.7321. Traditional models showed notable limitations; for example, naïve Bayes exhibited strong majority-class bias, despite an accuracy of 82.35% (Cohen’s kappa = 0.0054). Among the oversampling methods, SMOTE was the most effective in improving the fairness of traditional models, while RoBERTa’s fine-tuning process inherently mitigated class imbalance. A computational analysis highlights key trade-offs: traditional models train quickly but require oversampling, deep learning offers a balanced trade-off between performance and efficiency, and transformer models provide state-of-the-art accuracy at the cost of high computational resources. These findings offer evidence-based guidance for selecting appropriate models for tourism sentiment analysis.