A Multi-Hierarchical attention-based prediction method on Time Series with spatio-temporal context among variables

Multi-scale prediction of multivariate time series in Earth system science is a challenging problem due to the task with spatio-temporal context between multi-type variables. For example Offshore wind is influenced by the spatio-temporal features of own and other ocean elements. Existing methods do not fully exploit the spatio-temporal features and influence information of non-predictive series on target series, so the major challenge is how to effectively extract these features and integrate them into an end-to-end network. In this paper, we propose a multi-hierarchical attention network (MHA), which exploits triple attention mechanisms to capture the correlations of multi-type variables in identical spacetime, different space at the same time and different spacetime. The first hierarchical of attention captures the coupling mechanisms of multi-type variables at the same spacetime. The second hierarchical of attention extracts spatial relationships between target and non-predictive series at each moment. The third hierarchical of attention selects the relevant information across all time steps to learn the time dependence of data. Experiments on two different real-world datasets, viz., Offshore wind data and Ocean current data, demonstrate the effectiveness and robustness of the developed approach. Specifically, the triple attention can successfully capture internal patterns between variables in different spacetime. Overall, our proposed method not only improves the prediction performance of multivariate time series in Earth system science, but also reveals interaction patterns between variables from a data-driven perspective.