Real-time solar irradiance forecasting for grid integration using all-sky imagery and multi-stage AI with Kalman filter optimization

Rapid photovoltaic (PV) integration challenges grid stability under dynamic cloud conditions. Current forecasting methods are limited to 11–15 min horizons, rely on computationally intensive black-box models, and lack the accuracy balance required for operational grid management. This study introduces a novel forecasting framework providing accurate irradiance forecasts up to 30 min ahead while maintaining real-time computational efficiency. The operational framework integrates advanced sky image analysis with a hybrid AI architecture and Kalman filtering optimization. Key technical innovations include (1) superpixel-based cloud detection using Simple Linear Iterative Clustering (SLIC) for precise atmospheric characterization and (2) a hybrid Support Vector Machine–Convolutional Neural Network (SVM–CNN) model with Kalman filtering for Clear Sky Index estimation across diverse weather conditions. A weather-adaptive clustering module dynamically adjusts forecasting strategies across five sky conditions, while multi-frequency modeling captures spatial–temporal variability. Compared to state-of-the-art deep learning methods, the proposed framework demonstrates superior forecasting accuracy while requiring significantly fewer computational resources, making it suitable for deployment on edge devices and for real-time grid applications. Validation against measured data shows forecast skill (FS) improvements ranging from 8.3% to 22% and 6.7% to 18% over smart persistence benchmarks. Kalman filtering further reduces FS error by 20%, particularly under challenging sky conditions.