Weighted robust hybrid partial least squares regression forest

Partial Least Squares Regression (PLSR) effectively handles high-dimensional or multicollinear data, while Regression Forest (RF) excels at ensemble learning, nonlinearity, and robustness to large feature sets. Their integration into a hybrid PLSR-RF framework can further enhance predictive performance; however, both conventional PLSR and RF—along with the existing hybrid—remain sensitive to outliers. Although robust versions of PLSR and RF have been explored, a robustified hybrid approach is notably absent. To address this gap, we introduce the Weighted Robust Hybrid Partial Least Squares Regression Forests. Our method uses the robust iteratively reweighted SIMPLS algorithm to generate orthogonal components for forest construction, accompanied by an innovative weighting scheme, robust loss functions, and predictions to diminish outlier influence within each tree. We also propose an alternative bootstrap procedure that curtails outlier effects and reduces tree complexity. We compare the performance of the proposed hybrid approach with the classical and Robust PLS and ordinary and robust random forests. Extensive numerical studies demonstrate that, under various levels of contamination and across a range of error distributions, our hybrid methods consistently outperform classical and robust PLS approaches and conventional random forests even when no multicollinearity problem exists. These findings hold in moderate and high-dimensional settings, with real-data applications confirming the method’s resilience and versatility.