Applying machine learning to gauge the number of women in science, technology, and innovation policy (STIP): a model to accommodate missing data

The underrepresentation of women in science, technology, and innovation policy (STIP) continues to hinder global innovation and scientific advancement. While research has examined women’s participation in STEM and policymaking separately, their intersection within STIP as a distinct sector remains understudied. This study addresses this gap by developing a comprehensive machine learning framework to accurately measure and predict women’s representation in STIP while accounting for missing domestic data. Using data from 60 countries, we implemented hybrid machine learning models—including Linear Regression, ElasticNet, Lasso Regression, and Ridge Regression, and Support Vector Regression—to forecast women’s representation in STIP. The methodology incorporated advanced techniques such as K-Nearest Neighbors (KNN) imputation for missing data handling, feature engineering using autoencoders’ latent representations, and evaluation through multiple regression metrics. The SVR model achieved the highest predictive accuracy with an R2 score of 0.835, and mean cross validation score with standard deviation of 0.196 and the low error metrics (MAE: 0.2677; RMSE: 0.406), establishing the first reliable method for quantifying women’s representation in STIP as its own sector. Interestingly, countries without formal quota systems demonstrated superior performance metrics (median: 0.3675; mean: 0.3676) compared to those with quotas. Furthermore, the analysis revealed a critical disconnect between STEM participation rates and policymaking leadership roles, evidenced by a weak correlation (r = 0.11) between these domains. These findings challenge assumptions about the effectiveness of quota systems alone in driving gender equity and suggest that holistic approaches addressing systemic barriers are essential for sustainable progress. This study provides a novel analytical framework for evaluating gender representation in STIP while offering actionable insights into fostering gender equity across sectors. By delivering more accurate measurements of women’s representation in STIP, this research contributes significantly to evidence-based policymaking and establishes a foundation for future interdisciplinary studies on gender equity within this critical domain.