Robust regression under the general framework of bounded loss functions

Conventional regression methods often fail when encountering noise. The application of a bounded loss function is an effective means to enhance regressor robustness. However, most bounded loss functions exist in Ramp-style forms, losing some inherent properties of the original function due to hard truncation. Besides, there is currently no unified framework on how to design bounded loss functions. In response to the above two issues, this paper proposes a general framework that can smoothly and adaptively bound any non-negative function. It can not only degenerate to the original function, but also inherit its elegant properties, including symmetry, differentiability and smoothness. Under this framework, a robust regressor called bounded least squares support vector regression (BLSSVR) is proposed to mitigate the effects of noise and outliers by limiting the maximum loss. With appropriate parameters, the bounded least squares loss grows faster than its unbounded form in the initial stage, which facilitates BLSSVR to assign larger weights to non-outlier points. Meanwhile, the Nesterov accelerated gradient (NAG) algorithm is employed to optimize BLSSVR. Extensive experiments on synthetic and real-world datasets profoundly demonstrate the superiority of BLSSVR over benchmark methods.