IDEAS home Printed from https://ideas.repec.org/a/eee/energy/v322y2025ics036054422501179x.html
   My bibliography  Save this article

ShuffleTransformerMulti-headAttentionNet network for user load forecasting

Author

Listed:
  • Yin, Linfei
  • Ju, Linyi

Abstract

With the development of intelligent power load forecasting, optimizing and improving algorithms to increase the accuracy and speed of load forecasting is a key part of responding to policy development opportunities. However, since the load data is affected by many factors, the output is characterized by volatility, nonlinearity and non-smoothness, and the volume is large, and it is difficult to mine the hidden information, the stable operation and economic benefits of the power system will become a challenge. To improve data prediction accuracy and maintain power system stable operation, this study proposes a ShuffleTransformerMulti-headAttentionNet network model. The structural feature of the network model is that two CNN networks are input to a Transformer model in a parallel way. The ShuffleTransformerMulti-headAttentionNet network model is compared with 20 network models such as Shufflenet and GooleNet. To better improve the applicability of the model, we discuss small-scale, high temporal resolution and large-scale, low temporal resolution historical power load data. The feasibility and reliability of the ShuffleTransformerMulti-headAttentionNet algorithm is verified by simulating the input data of Case 1:292 × 96 and Case 2:31720 × 24, respectively. The experimental results show that the precision of the ShuffleTransformerMulti-headAttentionNet network is the highest in the comparison experiment, with root mean squared error, mean squared error, mean absolute error and symmetric mean absolute percentage error as the reference standards. In case 1, RMSE, MSE, MAE, MAPE, and SMAPE of ShuffleTransformerMulti-headAttentionNet network are optimized 20.27 %–100 %, 36.43 %–100 %, 11.51 %–100 %, 0.76 %–100 %, 5.97 %–96.00 % compared with other algorithms, respectively. In case 2, RMSE, MSE, MAE, MAPE, and SMAPE of ShuffleTransformerMulti-headAttentionNet network are optimized by 21.65 %–100 %, 38.62 %–100 %, 36.20 %–100 %, 12.72 %–100 %, 26.00 %–96.05 % compared with other algorithms, respectively.

Suggested Citation

  • Yin, Linfei & Ju, Linyi, 2025. "ShuffleTransformerMulti-headAttentionNet network for user load forecasting," Energy, Elsevier, vol. 322(C).
    • Handle: RePEc:eee:energy:v:322:y:2025:i:c:s036054422501179x
    DOI: 10.1016/j.energy.2025.135537
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S036054422501179X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.energy.2025.135537?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Zhang, Yi-Ming & Wang, Hao, 2023. "Multi-head attention-based probabilistic CNN-BiLSTM for day-ahead wind speed forecasting," Energy, Elsevier, vol. 278(PA).
    2. Xiang, Ling & Fu, Xiaomengting & Yao, Qingtao & Zhu, Guopeng & Hu, Aijun, 2024. "A novel model for ultra-short term wind power prediction based on Vision Transformer," Energy, Elsevier, vol. 294(C).
    3. Zang, Haixiang & Xu, Ruiqi & Cheng, Lilin & Ding, Tao & Liu, Ling & Wei, Zhinong & Sun, Guoqiang, 2021. "Residential load forecasting based on LSTM fusing self-attention mechanism with pooling," Energy, Elsevier, vol. 229(C).
    4. Wang, Shuai & Wang, Jianzhou & Lu, Haiyan & Zhao, Weigang, 2021. "A novel combined model for wind speed prediction – Combination of linear model, shallow neural networks, and deep learning approaches," Energy, Elsevier, vol. 234(C).
    5. Das, Kaushik & Kumar, Roushan & Krishna, Anurup, 2024. "Analyzing electric vehicle battery health performance using supervised machine learning," Renewable and Sustainable Energy Reviews, Elsevier, vol. 189(PA).
    6. Tian, Zhirui & Liu, Weican & Jiang, Wenqian & Wu, Chenye, 2024. "CNNs-Transformer based day-ahead probabilistic load forecasting for weekends with limited data availability," Energy, Elsevier, vol. 293(C).
    7. Dai, Yeming & Zhao, Pei, 2020. "A hybrid load forecasting model based on support vector machine with intelligent methods for feature selection and parameter optimization," Applied Energy, Elsevier, vol. 279(C).
    8. Li, Chuang & Li, Guojie & Wang, Keyou & Han, Bei, 2022. "A multi-energy load forecasting method based on parallel architecture CNN-GRU and transfer learning for data deficient integrated energy systems," Energy, Elsevier, vol. 259(C).
    9. Xu, Mengjie & Li, Qianwen & Zhao, Zhengtang & Sun, Chuanwang, 2024. "Bilinear-DRTFT: Uncertainty prediction in electricity load considering multiple demand responses," Energy, Elsevier, vol. 309(C).
    10. Wang, Yun & Zou, Runmin & Liu, Fang & Zhang, Lingjun & Liu, Qianyi, 2021. "A review of wind speed and wind power forecasting with deep neural networks," Applied Energy, Elsevier, vol. 304(C).
    11. Zhang, Yagang & Pan, Zhiya & Wang, Hui & Wang, Jingchao & Zhao, Zheng & Wang, Fei, 2023. "Achieving wind power and photovoltaic power prediction: An intelligent prediction system based on a deep learning approach," Energy, Elsevier, vol. 283(C).
    12. Jiang, Zongxi & Zhang, Luliang & Ji, Tianyao, 2023. "NSDAR: A neural network-based model for similar day screening and electric load forecasting," Applied Energy, Elsevier, vol. 349(C).
    13. Fan, Pengdan & Wang, Dan & Wang, Wei & Zhang, Xiuyu & Sun, Yuying, 2024. "A novel multi-energy load forecasting method based on building flexibility feature recognition technology and multi-task learning model integrating LSTM," Energy, Elsevier, vol. 308(C).
    14. Nascimento, Erick Giovani Sperandio & de Melo, Talison A.C. & Moreira, Davidson M., 2023. "A transformer-based deep neural network with wavelet transform for forecasting wind speed and wind energy," Energy, Elsevier, vol. 278(C).
    15. Safder, Usman & Hai, Tra Nguyen & Loy-Benitez, Jorge & Yoo, ChangKyoo, 2022. "Nationwide policymaking strategies to prevent future electricity crises in developing countries using data-driven forecasting and fuzzy-SWOT analyses," Energy, Elsevier, vol. 259(C).
    16. Li, Kang & Duan, Pengfei & Cao, Xiaodong & Cheng, Yuanda & Zhao, Bingxu & Xue, Qingwen & Feng, Mengdan, 2024. "A multi-energy load forecasting method based on complementary ensemble empirical model decomposition and composite evaluation factor reconstruction," Applied Energy, Elsevier, vol. 365(C).
    17. Niu, Dongxiao & Yu, Min & Sun, Lijie & Gao, Tian & Wang, Keke, 2022. "Short-term multi-energy load forecasting for integrated energy systems based on CNN-BiGRU optimized by attention mechanism," Applied Energy, Elsevier, vol. 313(C).
    18. Xu, Huifeng & Hu, Feihu & Liang, Xinhao & Zhao, Guoqing & Abugunmi, Mohammad, 2024. "A framework for electricity load forecasting based on attention mechanism time series depthwise separable convolutional neural network," Energy, Elsevier, vol. 299(C).
    19. Qiu, Tianzhi & Wang, Lunche & Lu, Yunbo & Zhang, Ming & Qin, Wenmin & Wang, Shaoqiang & Wang, Lizhe, 2022. "Potential assessment of photovoltaic power generation in China," Renewable and Sustainable Energy Reviews, Elsevier, vol. 154(C).
    20. Aslam, Sheraz & Herodotou, Herodotos & Mohsin, Syed Muhammad & Javaid, Nadeem & Ashraf, Nouman & Aslam, Shahzad, 2021. "A survey on deep learning methods for power load and renewable energy forecasting in smart microgrids," Renewable and Sustainable Energy Reviews, Elsevier, vol. 144(C).
    21. Fan, Jingmin & Zhong, Mingwei & Guan, Yuanpeng & Yi, Siqi & Xu, Cancheng & Zhai, Yanpeng & Zhou, Yongwang, 2024. "An online long-term load forecasting method: Hierarchical highway network based on crisscross feature collaboration," Energy, Elsevier, vol. 299(C).
    22. Yu, Min & Niu, Dongxiao & Gao, Tian & Wang, Keke & Sun, Lijie & Li, Mingyu & Xu, Xiaomin, 2023. "A novel framework for ultra-short-term interval wind power prediction based on RF-WOA-VMD and BiGRU optimized by the attention mechanism," Energy, Elsevier, vol. 269(C).
    23. Yakoub, Ghali & Mathew, Sathyajith & Leal, Joao, 2023. "Intelligent estimation of wind farm performance with direct and indirect ‘point’ forecasting approaches integrating several NWP models," Energy, Elsevier, vol. 263(PD).
    24. Shi, Jian & Teh, Jiashen & Alharbi, Bader & Lai, Ching-Ming, 2024. "Load forecasting for regional integrated energy system based on two-phase decomposition and mixture prediction model," Energy, Elsevier, vol. 297(C).
    25. Kim, Jimin & Obregon, Josue & Park, Hoonseok & Jung, Jae-Yoon, 2024. "Multi-step photovoltaic power forecasting using transformer and recurrent neural networks," Renewable and Sustainable Energy Reviews, Elsevier, vol. 200(C).
    26. Zhong, Mingwei & Xu, Cancheng & Xian, Zikang & He, Guanglin & Zhai, Yanpeng & Zhou, Yongwang & Fan, Jingmin, 2024. "DTTM: A deep temporal transfer model for ultra-short-term online wind power forecasting," Energy, Elsevier, vol. 286(C).
    27. Chen, Xin & Ye, Xiaoling & Xiong, Xiong & Zhang, Yingchao & Li, Yuanlu, 2024. "Improving the accuracy of wind speed spatial interpolation: A pre-processing algorithm for wind speed dynamic time warping interpolation," Energy, Elsevier, vol. 295(C).
    28. Shi, Jian & Teh, Jiashen, 2024. "Load forecasting for regional integrated energy system based on complementary ensemble empirical mode decomposition and multi-model fusion," Applied Energy, Elsevier, vol. 353(PB).
    29. liu, Qian & li, Yulin & jiang, Hang & chen, Yilin & zhang, Jiang, 2024. "Short-term photovoltaic power forecasting based on multiple mode decomposition and parallel bidirectional long short term combined with convolutional neural networks," Energy, Elsevier, vol. 286(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhao, Xiaoyu & Duan, Pengfei & Cao, Xiaodong & Xue, Qingwen & Zhao, Bingxu & Hu, Jinxue & Zhang, Chenyang & Yuan, Xiaoyang, 2025. "A probabilistic load forecasting method for multi-energy loads based on inflection point optimization and integrated feature screening," Energy, Elsevier, vol. 327(C).
    2. Hu, Rong & Zhou, Kaile & Lu, Xinhui, 2025. "Integrated loads forecasting with absence of crucial factors," Energy, Elsevier, vol. 322(C).
    3. Peng, Daogang & Liu, Yu & Wang, Danhao & Zhao, Huirong & Qu, Bogang, 2024. "Multi-energy load forecasting for integrated energy system based on sequence decomposition fusion and factors correlation analysis," Energy, Elsevier, vol. 308(C).
    4. Fan, Jingmin & Zhong, Mingwei & Guan, Yuanpeng & Yi, Siqi & Xu, Cancheng & Zhai, Yanpeng & Zhou, Yongwang, 2024. "An online long-term load forecasting method: Hierarchical highway network based on crisscross feature collaboration," Energy, Elsevier, vol. 299(C).
    5. Ren, Xiaoxiao & Tian, Xin & Wang, Kai & Yang, Sifan & Chen, Weixiong & Wang, Jinshi, 2025. "Enhanced load forecasting for distributed multi-energy system: A stacking ensemble learning method with deep reinforcement learning and model fusion," Energy, Elsevier, vol. 319(C).
    6. Wang, Danhao & Peng, Daogang & Huang, Dongmei & Zhao, Huirong & Qu, Bogang, 2025. "MMEMformer: A multi-scale memory-enhanced transformer framework for short-term load forecasting in integrated energy systems," Energy, Elsevier, vol. 322(C).
    7. Chen, Haoyu & Huang, Hai & Zheng, Yong & Yang, Bing, 2024. "A load forecasting approach for integrated energy systems based on aggregation hybrid modal decomposition and combined model," Applied Energy, Elsevier, vol. 375(C).
    8. Liao, Chengchen & Tan, Mao & Li, Kang & Chen, Jie & Wang, Rui & Su, Yongxin, 2024. "Sequence signal prediction and reconstruction for multi-energy load forecasting in integrated energy systems: A bi-level multi-task learning method," Energy, Elsevier, vol. 313(C).
    9. Hu, Likun & Cao, Yi & Yin, Linfei, 2024. "Long-term price guidance mechanism for integrated energy systems based on gated recurrent unit - vision transformer prediction and fractional-order stochastic dynamic calculus control," Energy, Elsevier, vol. 312(C).
    10. Wang, Jianguo & Han, Lincheng & Zhang, Xiuyu & Wang, Yingzhou & Zhang, Shude, 2023. "Electrical load forecasting based on variable T-distribution and dual attention mechanism," Energy, Elsevier, vol. 283(C).
    11. Wang, Shuangxin & Shi, Jiarong & Yang, Wei & Yin, Qingyan, 2024. "High and low frequency wind power prediction based on Transformer and BiGRU-Attention," Energy, Elsevier, vol. 288(C).
    12. Sun, Yang & Tian, Zhirui, 2025. "Solving few-shot problem in wind speed prediction: A novel transfer strategy based on decomposition and learning ensemble," Applied Energy, Elsevier, vol. 377(PD).
    13. Xie, Xiangmin & Ding, Yuhao & Sun, Yuanyuan & Zhang, Zhisheng & Fan, Jianhua, 2024. "A novel time-series probabilistic forecasting method for multi-energy loads," Energy, Elsevier, vol. 306(C).
    14. Shi, Jian & Teh, Jiashen & Lai, Ching-Ming, 2025. "Wind power prediction based on improved self-attention mechanism combined with Bi-directional Temporal Convolutional Network," Energy, Elsevier, vol. 322(C).
    15. Fan, Pengdan & Wang, Dan & Wang, Wei & Zhang, Xiuyu & Sun, Yuying, 2024. "A novel multi-energy load forecasting method based on building flexibility feature recognition technology and multi-task learning model integrating LSTM," Energy, Elsevier, vol. 308(C).
    16. Li, Jianfang & Jia, Li & Zhou, Chengyu, 2024. "Probability density function based adaptive ensemble learning with global convergence for wind power prediction," Energy, Elsevier, vol. 312(C).
    17. Xu, Huifeng & Hu, Feihu & Liang, Xinhao & Zhao, Guoqing & Abugunmi, Mohammad, 2024. "A framework for electricity load forecasting based on attention mechanism time series depthwise separable convolutional neural network," Energy, Elsevier, vol. 299(C).
    18. Zhang, Chu & Li, Zhengbo & Ge, Yida & Liu, Qianlong & Suo, Leiming & Song, Shihao & Peng, Tian, 2024. "Enhancing short-term wind speed prediction based on an outlier-robust ensemble deep random vector functional link network with AOA-optimized VMD," Energy, Elsevier, vol. 296(C).
    19. Wang, Yong & Fan, Neng & Wen, Shixiong & Kuang, Wenyu & Yang, Zhongsen & Xiao, Wenlian & Li, Hong-Li & Narayanan, Govindasami & Sapnken, Flavian Emmanuel, 2025. "A novel structural adaptive discrete grey Euler model and its application in clean energy production and consumption," Energy, Elsevier, vol. 323(C).
    20. Wen, Qianyun & Liu, Yang, 2025. "Feature engineering and selection for prosumer electricity consumption and production forecasting: A comprehensive framework," Applied Energy, Elsevier, vol. 381(C).

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:energy:v:322:y:2025:i:c:s036054422501179x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/energy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.