Machine learning-based prediction models for electoral outcomes in India: a comparative analysis of exit polls from 2014–2021

Political researchers and scientists have long sought to forecast the results of upcoming elections in a democratic setup. Prediction approaches have been combined to improve prediction accuracy. In this work, we focus on India, utilizing exit polls spanning the last 8 years to develop machine learning-based prediction models. Exit polls provided by major media agencies for both provincial as well as the union legislative assemblies have been used as inputs to improve prediction accuracy. The training set comprises of the seats projected for the United Progressive Alliance (UPA) and the National Democratic Alliance (NDA) in state and central government elections from 2014 through 2020, providing ample data for training across a wide range of short and long-term trends. The testing set includes seats projected for the UPA and the NDA in 2021, across states including Assam, Kerala, Pondicherry, Tamil Nadu and West Bengal, ensuring validation of the prediction models using undisclosed ‘out of sample’ data. Trained models have been used to provide projected seats for the 2021 legislative assembly elections in the states of Assam, Kerala, Pondicherry, Tamil Nadu and West Bengal. Efficient machine learning approaches viz. support vector regression (SVR), extreme gradient boosting (XGBoost) and deep CNN-LSTM networks have been explored as predictors. The performance of the predictive models is evaluated by utilizing the metrics such as relative error (RE%), normalised mean square error (NRMSE), fractional bias (FB), coefficient of determination (R2) and normalised absolute error (NAE). XGBoost exhibits the lowest R2 value of 0.97, indicating superior fit to the observed data compared to other methods. Additionally, it demonstrates the lowest FB value of −0.03, suggesting minimal prediction bias. Notably, XGBoost also yields the lowest NRMSE, NAE, RE% values (4.11, 2.73 and 0.97% respectively), highlighting its superior accuracy in predicting election outcomes. XGBoost is followed by the deep CNN-LSTM model, while the SVR model exhibits the lowest prediction accuracy among the proposed forecasting methods.