A high-dimensional classifier with variable selection using mirror statistics

In this work, we propose a new classifier based on Fisher’s linear discriminant analysis (LDA), which is a two-stage procedure serving variable selection and classification tasks. The variable selection scheme is to select covariates that belong to the discriminative set, and this approach is aimed at obtaining a better classifier rather than choosing significant variables themselves. In the first stage, we adopt a notion of mirror statistic proposed recently in Xing, Zhao, and Liu (2023), and the direction vector is obtained by a regularized form of the sample covariance matrix and a James-Stein type estimator for the mean vectors. In the second stage, we developed a new classifier with selected variables in a meticulous way. We implemented a modified ϵ-greedy algorithm empirically. A broad range of simulation studies and a real-data analysis are conducted to show the better performance of our proposed classifier in terms of classification accuracy and variable selection.