Nowcasting Solar Irradiance Components Using a Vision Transformer and Multimodal Data from All-Sky Images and Meteorological Observations

As the solar share in energy generation is expanding globally, solar nowcasting is becoming increasingly important for the efficient and economical management of the power grid. This study leveraged the spatial context provided by all-sky images (ASI) in addition to the meteorological records for improved nowcasting of global, direct, and diffuse irradiance components. The proposed methodology consists of two branches for processing the multimodal data of ASIs and meteorological data. Due to its capability of understanding the overall characteristics of the image through self-attention, a vision transformer is utilized for the image branch while normal dense layers process the tabular meteorological data. The proposed architecture is compared against the baselines of the Ineichen clear sky model, a feedforward neural network (FFNN) where cloud coverage is computed from the ASIs by a simple color-channel threshold algorithm, and a hybrid of FFNN and U-Net model, which replaces the color threshold algorithm with fully convolutional layers for cloud segmentation. The models are trained, validated, and tested using the quality-assured ground-truth data collected in Ulaanbaatar, Mongolia, from May to August 2024, under one-minute intervals with a random split of 70%, 15%, and 15%. Our approach exhibits superior performance to baselines with a significantly lower mean absolute error (MAE) of 15–33 W/m 2 and root mean square error (RMSE) of 26–72 W/m 2 , thus potentially aiding grid operators’ decision-making in real-time.