EnergyGPT: Fine-tuning large language model for multi-energy load forecasting

Multi-energy load forecasting plays a pivotal role in strategy management and operational planning in integrated energy systems (IESs). However, the inherent complexity and high dynamicity of IESs bring significant challenges, including modeling the coupling relationships between relevant features and designing effective forecasting models. The traditional statistical methods for feature correlation analysis struggle to reveal the dynamic correlation of feature sequences, and the classic data-driven machine learning methods for multi-energy load forecasting suffer from a lack of generalization. To address these issues, inspired by the successful applications of pre-trained large models in specific tasks, this paper introduces a large language model (LLM) into the energy field and proposes a novel multi-energy load forecasting model based on GPT2, called EnergyGPT, which utilizes the knowledge and capabilities of LLMs trained on a massive text stream corpus to enhance generalization and performance in multi-energy load forecasting. Additionally, EnergyGPT adopts a dynamic self-attention module to adjust the importance of sequential features. The proposed EnergyGPT is evaluated on the real load data from the Tempe campus provided by Arizona State University’s online platform, compared with a set of the latest load forecasting methods. The experimental results show that EnergyGPT outperforms all the comparisons with the best performance in four seasons, which validates the effectiveness of EnergyGPT and shows the potential of large models for cross-domain applications.