Spatial–temporal multimodal fusion model for intra-hour solar power forecasting under variable weather conditions

Intra-hour solar power forecasting (IHSPF) is crucial for handling the big variation of solar energy. Conventional IHSPF methods either overlook historical photovoltaic (PV) power generation data or neglect the interaction between spatial and temporal information. Additionally, few methods consider the weather diversity that challenges accurate power forecasting. To solve these problem, this paper proposes a Spatial–Temporal Multimodal Fusion Model (STMFM) for IHSPF under variable weather conditions. STMFM leverages both historical sky image sequences and PV power generation data. It employs a dual-stream structure for comprehensive feature extraction. An Adaptive Fusion Module (AFM) is introduced to optimize the combination of multimodal features and establish the interaction between spatial and temporal information. A temporal inference module captures hidden temporal relationships, and a weather dependent decoder (WDD) is proposed to enhance the model’s adaptability to diverse weather conditions by dynamically adjusting parameters based on historical weather information. Extensive experimental results on public datasets demonstrate high effectiveness and strong generalization capability of the proposed STMFM, which outperforms baseline methods by 5.66%–22.81% of Forecast skill (FS) at the 15-min forecasting horizon.