Multi-modal feature fusion model based on TimesNet and T2T-ViT for ultra-short-term solar irradiance forecasting

Solar power generation is considered a solution to meet global energy needs. Accurate solar energy prediction can provide a basis for the stable operation and economic dispatch of power systems. Although the solar irradiance prediction method based on historical data and sky images has been widely studied, the exploration of mining deep time series and image features and associating the two features for effective modeling is still limited. Therefore, this paper proposes a multi-modal feature learning model based on TimesNet and T2T-ViT for ultra-short-term solar irradiance prediction. Firstly, the historical sequence is transformed into a two-dimensional tensor using TimesNet, and the temporal features are extracted using two-dimensional convolution. Secondly, T2T-ViT is used to model the global information and local structure, and the deep image features are extracted. Finally, a feature fusion module based on Transformer is proposed. Image features enhance the temporal features, and the decoder is used to output the prediction results of the next six steps (1 h in advance, the prediction step is 10 min). The experimental results show that the proposed method has better prediction performance than other SOTA methods, and has good robustness in the whole prediction range.