IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v13y2025i6p996-d1615104.html
   My bibliography  Save this article

Mathematical Methods in Feature Selection: A Review

Author

Listed:
  • Firuz Kamalov

    (Department of Electrical Engineering, Canadian University Dubai, Dubai P.O. Box 117781, United Arab Emirates)

  • Hana Sulieman

    (Department of Mathematics and Statistics, American University of Sharjah, Sharjah P.O. Box 26666, United Arab Emirates)

  • Ayman Alzaatreh

    (Department of Mathematics and Statistics, American University of Sharjah, Sharjah P.O. Box 26666, United Arab Emirates)

  • Maher Emarly

    (Department of Mathematics and Statistics, American University of Sharjah, Sharjah P.O. Box 26666, United Arab Emirates)

  • Hasna Chamlal

    (Computer Science and Systems Laboratory (LIS), Faculty of Sciences Ain Chock, Hassan II University of Casablanca, Casablanca 20360, Morocco)

  • Murodbek Safaraliev

    (Ural Power Engineering Institute, Ural Federal University, Yekaterinburg 620002, Russia)

Abstract

Feature selection is essential in machine learning and data science. Recently, there has been a growing effort to apply various mathematical methods to construct novel feature selection algorithms. In this study, we present a comprehensive state-of-the-art review of such techniques. We propose a new mathematical framework-based taxonomy to group the existing literature and provide an analysis of the research in each category from a mathematical perspective. The key frameworks discussed include variance-based methods, regularization methods, and Bayesian methods. By analyzing the strengths and limitations of each technique, we provide insights into their applicability across various domains. The review concludes with emerging trends and future research directions for mathematical methods in feature selection.

Suggested Citation

  • Firuz Kamalov & Hana Sulieman & Ayman Alzaatreh & Maher Emarly & Hasna Chamlal & Murodbek Safaraliev, 2025. "Mathematical Methods in Feature Selection: A Review," Mathematics, MDPI, vol. 13(6), pages 1-29, March.
    • Handle: RePEc:gam:jmathe:v:13:y:2025:i:6:p:996-:d:1615104
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/13/6/996/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/13/6/996/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Zou, Hui, 2006. "The Adaptive Lasso and Its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 1418-1429, December.
    2. Emre Barut & Jianqing Fan & Anneleen Verhasselt, 2016. "Conditional Sure Independence Screening," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(515), pages 1266-1277, July.
    3. Wentao Wang & Jiaxuan Liang & Rong Liu & Yunquan Song & Min Zhang, 2022. "A Robust Variable Selection Method for Sparse Online Regression via the Elastic Net Penalty," Mathematics, MDPI, vol. 10(16), pages 1-18, August.
    4. Oyebayo Ridwan Olaniran & Aliu Omotayo Sikiru & Jeza Allohibi & Abdulmajeed Atiah Alharbi & Nada MohammedSaeed Alharbi, 2025. "Hybrid Random Feature Selection and Recurrent Neural Network for Diabetes Prediction," Mathematics, MDPI, vol. 13(4), pages 1-25, February.
    5. Runze Li & Wei Zhong & Liping Zhu, 2012. "Feature Screening via Distance Correlation Learning," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 107(499), pages 1129-1139, September.
    6. Fan J. & Li R., 2001. "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1348-1360, December.
    7. Yang Song & Jing Zhang & Yu-dong Zhang & Ying Hou & Xu Yan & Yida Wang & Minxiong Zhou & Ye-feng Yao & Guang Yang, 2020. "FeAture Explorer (FAE): A tool for developing and comparing radiomics models," PLOS ONE, Public Library of Science, vol. 15(8), pages 1-10, August.
    8. Ning Hao & Yang Feng & Hao Helen Zhang, 2018. "Model Selection for High-Dimensional Quadratic Regression via Regularization," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(522), pages 615-625, April.
    9. Hyonho Chun & Sündüz Keleş, 2010. "Sparse partial least squares regression for simultaneous dimension reduction and variable selection," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 72(1), pages 3-25, January.
    10. Rohan Mitra & Eyad Ali & Dara Varam & Hana Sulieman & Firuz Kamalov, 2024. "Variable Selection in Data Analysis: A Synthetic Data Toolkit," Mathematics, MDPI, vol. 12(4), pages 1-29, February.
    11. Jianqing Fan & Jinchi Lv, 2008. "Sure independence screening for ultrahigh dimensional feature space," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 70(5), pages 849-911, November.
    12. Hui Zou & Trevor Hastie, 2005. "Addendum: Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(5), pages 768-768, November.
    13. Hui Zou & Trevor Hastie, 2005. "Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(2), pages 301-320, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wang, Christina Dan & Chen, Zhao & Lian, Yimin & Chen, Min, 2022. "Asset selection based on high frequency Sharpe ratio," Journal of Econometrics, Elsevier, vol. 227(1), pages 168-188.
    2. Loann David Denis Desboulets, 2018. "A Review on Variable Selection in Regression Analysis," Econometrics, MDPI, vol. 6(4), pages 1-27, November.
    3. Jingxuan Luo & Lili Yue & Gaorong Li, 2023. "Overview of High-Dimensional Measurement Error Regression Models," Mathematics, MDPI, vol. 11(14), pages 1-22, July.
    4. Min Chen & Yimin Lian & Zhao Chen & Zhengjun Zhang, 2017. "Sure explained variability and independence screening," Journal of Nonparametric Statistics, Taylor & Francis Journals, vol. 29(4), pages 849-883, October.
    5. Aneiros, Germán & Novo, Silvia & Vieu, Philippe, 2022. "Variable selection in functional regression models: A review," Journal of Multivariate Analysis, Elsevier, vol. 188(C).
    6. Li, Xingxiang & Cheng, Guosheng & Wang, Liming & Lai, Peng & Song, Fengli, 2017. "Ultrahigh dimensional feature screening via projection," Computational Statistics & Data Analysis, Elsevier, vol. 114(C), pages 88-104.
    7. Liming Wang & Xingxiang Li & Xiaoqing Wang & Peng Lai, 2022. "Unified mean-variance feature screening for ultrahigh-dimensional regression," Computational Statistics, Springer, vol. 37(4), pages 1887-1918, September.
    8. Qinqin Hu & Lu Lin, 2022. "Feature Screening in High Dimensional Regression with Endogenous Covariates," Computational Economics, Springer;Society for Computational Economics, vol. 60(3), pages 949-969, October.
    9. Christis Katsouris, 2023. "High Dimensional Time Series Regression Models: Applications to Statistical Learning Methods," Papers 2308.16192, arXiv.org.
    10. Sheng, Ying & Wang, Qihua, 2020. "Model-free feature screening for ultrahigh dimensional classification," Journal of Multivariate Analysis, Elsevier, vol. 178(C).
    11. Xiang-Jie Li & Xue-Jun Ma & Jing-Xiao Zhang, 2017. "Robust feature screening for varying coefficient models via quantile partial correlation," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 80(1), pages 17-49, January.
    12. Li-Pang Chen, 2021. "Feature screening based on distance correlation for ultrahigh-dimensional censored data with covariate measurement error," Computational Statistics, Springer, vol. 36(2), pages 857-884, June.
    13. Peter Bühlmann & Jacopo Mandozzi, 2014. "High-dimensional variable screening and bias in subsequent inference, with an empirical comparison," Computational Statistics, Springer, vol. 29(3), pages 407-430, June.
    14. Lee, Ji Hyung & Shi, Zhentao & Gao, Zhan, 2022. "On LASSO for predictive regression," Journal of Econometrics, Elsevier, vol. 229(2), pages 322-349.
    15. Christidis, Anthony-Alexander & Van Aelst, Stefan & Zamar, Ruben, 2025. "Multi-model subset selection," Computational Statistics & Data Analysis, Elsevier, vol. 203(C).
    16. Tan, Xin Lu, 2019. "Optimal estimation of slope vector in high-dimensional linear transformation models," Journal of Multivariate Analysis, Elsevier, vol. 169(C), pages 179-204.
    17. Huicong Yu & Jiaqi Wu & Weiping Zhang, 2024. "Simultaneous subgroup identification and variable selection for high dimensional data," Computational Statistics, Springer, vol. 39(6), pages 3181-3205, September.
    18. Ricardo P. Masini & Marcelo C. Medeiros & Eduardo F. Mendes, 2023. "Machine learning advances for time series forecasting," Journal of Economic Surveys, Wiley Blackwell, vol. 37(1), pages 76-111, February.
    19. Pei Wang & Shunjie Chen & Sijia Yang, 2022. "Recent Advances on Penalized Regression Models for Biological Data," Mathematics, MDPI, vol. 10(19), pages 1-24, October.
    20. Chen, Shi & Härdle, Wolfgang Karl & López Cabrera, Brenda, 2018. "Regularization Approach for Network Modeling of German Energy Market," IRTG 1792 Discussion Papers 2018-017, Humboldt University of Berlin, International Research Training Group 1792 "High Dimensional Nonstationary Time Series".

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:13:y:2025:i:6:p:996-:d:1615104. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.