Outer-product-of-gradient approach based on random forests Kernel and its application for classification

In the classification task, high dimensionality and class imbalance pose significant challenges to enhancing both efficiency and accuracy. To tackle the high-dimensionality issue, we have devised the Outer-Product-of-Gradient (OPG) method for Sufficient Dimension Reduction (SDR), specifically utilizing a Kernel based on Random Forests (KeRF). Additionally, to address class imbalance, we integrate multiple class-imbalance learning techniques, including XGBoost, cost-sensitive SVM (CS-SVM), and fuzzy SVM for class imbalance learning (FSVM-CIL). Our experiments were conducted on datasets encompassing both balanced and unbalanced sample distributions. The results reveal that the OPG method utilizing KeRF compares favorably in computational speed to the original OPG method, which employs the straightforward ROT law for selecting the bandwidth of the Gaussian kernel. In terms of classification results, for balanced datasets, both OPG methods exhibit comparable performance and outperform other SDR methods. For imbalanced datasets, the OPG-KeRF method combined with class-imbalance learners achieves superior performance, attaining higher average PR-AUC values with smaller standard deviations compared to alternative SDR methods.