A comparison between penalized logistic regressions and classification tree approaches

The penalized models have been established to be necessary for classifying the emerging complex, high-dimensional datasets. One of these penalized models is the “modified elastic-net” (M-Enet). The special cases of M-Enet (ridge, bridge, Lasso, and elastic-net) can be produced based on its penalty function. By using the fact that the Fisher-Scoring (FS) and Newton-Raphson (NR) algorithms use different approaches to solve numerical problems but are the same in the canonical link, in order to ease implementation, we developed an FS algorithm for the computation of the parameter estimates of the M-Enet. A simulation study of the microbiome as complex, high-dimensional data shows that each special case of the M-Enet penalty function has superiority in the sense of model accuracy, sensitivity, and feature selection technique. A comparison study is performed between the classification of binary responses using M-Enet as a parametric method and the classification tree via “GUIDE” as a non parametric method. The statistical analysis of the numerical results reveals that the special cases of M-Enet models outperform other M-Enet models and GUIDE in terms of average prediction accuracy and sensitivity, with p-values of 0. 0362 and 0. 0214, respectively. Consequently, these models exhibit superior classification performance compared to other classifier models.