Unveiling structural change determinants: A machine learning approach to long-term dynamics

This research aims to analyze the determinants of structural change (SC) between 2000 and 2021 by solving a classification problem via a novel combination of unsupervised and supervised machine learning (ML) techniques. These techniques facilitate training two binary logistic algorithms (LAs) that predict countries' long-term latent tendencies toward structural change (SC). The ML techniques employed in this study included principal component analysis (PCA), the validation set (VS) approach, the resampling approach, and the training of two benchmark algorithms to assess the trade-off between interpretability and prediction accuracy. In addition, supportive ML techniques including feature selection (FS), SHAP (SHapley additive explanations) values, the Lorenz Zonoid-based approach, and regularization, were used to enhance interpretability and model refinement. The findings demonstrate the empirical relevance of the SC's system approach and the predictors' potential to trigger cumulative causation mechanisms that engender systemic transformations and predict the long-term trends of countries toward an SC process or its stagnation and decline. The metrics indicate that the LAs demonstrate a notable capacity for prediction and classification, with a range of prediction accuracies from 0.87 to 0.97, an area under the receiver operating characteristic curve from 0.93 to 0.96, and a Youden index from 0.79 to 0.93. The study's findings offer empirical, actionable, and methodological implications for the SC field.