Neural Network Ensembles for Univariate Time Series Forecasting

Forecast combinations are considered a standard practice in many time series forecasting tasks, due to their documented success in improving the accuracy and robustness of the final forecasts. Regardless of the chosen combination scheme, constructing an ensemble of models reduces the impact of individual models’ biases and the need for selecting a single best model. These potential benefits are even more critical when forecasting neural networks are involved, as their use introduces new challenges, mainly linked with their stochastic nature and the large number of hyper-parameters influencing their performance. Motivated by the widespread adoption of neural networks in forecasting applications, in this paper we explore in greater detail the combination of forecasts produced by ensembles of feed-forward networks. We focus on the impact that different initialization seeds and “high-level” parameters, such as the size of the input vector and the loss functionLoss function, have on forecasting accuracy. We empirically evaluate the performance of individual models and ensembles of models, using three sets of series from the M4 competition. Our results suggest that ensembling neural networks significantly boosts forecasting performanceForecasting performance, but at the costCost of additional computational time.