Clagging: an efficient alternative to bagging

We introduce a new forecast combination strategy called clagging (for cluster aggregating), which consists in combining models fitted on different clusters. First, we perform K clustering tasks of the same training set, increasing the number of clusters from 1 to K. Next, we fit a model on each of those 1 + 2 +. . . + K clusters. Finally, the aggregate forecast for a new observation is obtained by combining the forecasts of the corresponding models using the distance of the new observation to the clusters’ centroids. We perform an extensive horse race study where we benchmark clagging on 20 datasets using 7 prediction models, considering both regression and classification tasks. Our results suggest that clagging outperforms bagging, where a bootstrapped sample is traditionally created by drawing observations with replacement until the size of the bootstrapped sample coincides with the size of the original training set. Clagging also improve the performance compared to a standard fit on the whole training set.