IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v327y2022ics0306261922013204.html
   My bibliography  Save this article

A deep model for short-term load forecasting applying a stacked autoencoder based on LSTM supported by a multi-stage attention mechanism

Author

Listed:
  • Fazlipour, Zahra
  • Mashhour, Elaheh
  • Joorabian, Mahmood

Abstract

This paper presents an innovative univariate Deep LSTM-based Stacked Autoencoder (DLSTM-SAE) model for short-term load forecasting, equipped with a Multi-Stage Attention Mechanism (MSAM), including an input AM and several temporal AM in the pre-training phase. The input AM is used to capture the high-impact load sequence time steps of univariate input data. It should be noted that the model's performance is improved by increasing the network depth; however, finding the optimal network parameters is a challenging task due to the random assignment of the initial weights of the network. An unsupervised greedy layer-wise pre-training structure equipped with the MSAM is expanded to solve setting the random initial weight problem of the DLSTM-SAE model. The multi-stage temporal AM in the pre-training structure leads the DLSTM-SAE to properly learn the time dependencies related to remarkably long sequence input data and capture the temporal merit features lied in the LSTM memory. The performance of the proposed model is evaluated through various comparative tests with current prevalent models using actual energy market data New England ISO using three criteria indexes. The results show the superiority of the proposed model and its robustness in offline and online load forecasting.

Suggested Citation

  • Fazlipour, Zahra & Mashhour, Elaheh & Joorabian, Mahmood, 2022. "A deep model for short-term load forecasting applying a stacked autoencoder based on LSTM supported by a multi-stage attention mechanism," Applied Energy, Elsevier, vol. 327(C).
    • Handle: RePEc:eee:appene:v:327:y:2022:i:c:s0306261922013204
    DOI: 10.1016/j.apenergy.2022.120063
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261922013204
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2022.120063?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Xiao, Liye & Wang, Jianzhou & Hou, Ru & Wu, Jie, 2015. "A combined model based on data pre-analysis and weight coefficients optimization for electrical load forecasting," Energy, Elsevier, vol. 82(C), pages 524-549.
    2. Xu, Lei & Wang, Shengwei & Tang, Rui, 2019. "Probabilistic load forecasting for buildings considering weather forecasting uncertainty and uncertain peak load," Applied Energy, Elsevier, vol. 237(C), pages 180-195.
    3. Sadaei, Hossein Javedani & de Lima e Silva, Petrônio Cândido & Guimarães, Frederico Gadelha & Lee, Muhammad Hisyam, 2019. "Short-term load forecasting by using a combined method of convolutional neural networks and fuzzy time series," Energy, Elsevier, vol. 175(C), pages 365-377.
    4. Xue, Guixiang & Qi, Chengying & Li, Han & Kong, Xiangfei & Song, Jiancai, 2020. "Heating load prediction based on attention long short term memory: A case study of Xingtai," Energy, Elsevier, vol. 203(C).
    5. Wen, Lulu & Zhou, Kaile & Yang, Shanlin & Lu, Xinhui, 2019. "Optimal load dispatch of community microgrid with deep learning based solar power and load forecasting," Energy, Elsevier, vol. 171(C), pages 1053-1065.
    6. Li, Yanying & Che, Jinxing & Yang, Youlong, 2018. "Subsampled support vector regression ensemble for short term electric load forecasting," Energy, Elsevier, vol. 164(C), pages 160-170.
    7. Zang, Haixiang & Xu, Ruiqi & Cheng, Lilin & Ding, Tao & Liu, Ling & Wei, Zhinong & Sun, Guoqiang, 2021. "Residential load forecasting based on LSTM fusing self-attention mechanism with pooling," Energy, Elsevier, vol. 229(C).
    8. Jain, Rishee K. & Smith, Kevin M. & Culligan, Patricia J. & Taylor, John E., 2014. "Forecasting energy consumption of multi-family residential buildings using support vector regression: Investigating the impact of temporal and spatial monitoring granularity on performance accuracy," Applied Energy, Elsevier, vol. 123(C), pages 168-178.
    9. Chen, Yibo & Tan, Hongwei, 2017. "Short-term prediction of electric demand in building sector via hybrid support vector regression," Applied Energy, Elsevier, vol. 204(C), pages 1363-1374.
    10. Zhao, Xinyu & Bai, Mingliang & Yang, Xusheng & Liu, Jinfu & Yu, Daren & Chang, Juntao, 2021. "Short-term probabilistic predictions of wind multi-parameter based on one-dimensional convolutional neural network with attention mechanism and multivariate copula distribution estimation," Energy, Elsevier, vol. 234(C).
    11. Lusis, Peter & Khalilpour, Kaveh Rajab & Andrew, Lachlan & Liebman, Ariel, 2017. "Short-term residential load forecasting: Impact of calendar effects and forecast granularity," Applied Energy, Elsevier, vol. 205(C), pages 654-669.
    12. Qu, Jiaqi & Qian, Zheng & Pei, Yan, 2021. "Day-ahead hourly photovoltaic power forecasting using attention-based CNN-LSTM neural network embedded with multiple relevant and target variables prediction pattern," Energy, Elsevier, vol. 232(C).
    13. Imani, Maryam, 2021. "Electrical load-temperature CNN for residential load forecasting," Energy, Elsevier, vol. 227(C).
    14. Jurado, Sergio & Nebot, Àngela & Mugica, Fransisco & Avellana, Narcís, 2015. "Hybrid methodologies for electricity load forecasting: Entropy-based feature selection with machine learning and soft computing techniques," Energy, Elsevier, vol. 86(C), pages 276-291.
    15. Rafati, Amir & Joorabian, Mahmood & Mashhour, Elaheh, 2020. "An efficient hour-ahead electrical load forecasting method based on innovative features," Energy, Elsevier, vol. 201(C).
    16. Vaghefi, A. & Jafari, M.A. & Bisse, Emmanuel & Lu, Y. & Brouwer, J., 2014. "Modeling and forecasting of cooling and electricity load demand," Applied Energy, Elsevier, vol. 136(C), pages 186-196.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Zhuang, Dian & Gan, Vincent J.L. & Duygu Tekler, Zeynep & Chong, Adrian & Tian, Shuai & Shi, Xing, 2023. "Data-driven predictive control for smart HVAC system in IoT-integrated buildings with time-series forecasting and reinforcement learning," Applied Energy, Elsevier, vol. 338(C).
    2. Zhang, Yagang & Wang, Hui & Wang, Jingchao & Cheng, Xiaodan & Wang, Tong & Zhao, Zheng, 2024. "Ensemble optimization approach based on hybrid mode decomposition and intelligent technology for wind power prediction system," Energy, Elsevier, vol. 292(C).
    3. Zhou, Guangzhao & Guo, Zanquan & Sun, Simin & Jin, Qingsheng, 2023. "A CNN-BiGRU-AM neural network for AI applications in shale oil production prediction," Applied Energy, Elsevier, vol. 344(C).
    4. Wei, Nan & Yin, Chuang & Yin, Lihua & Tan, Jingyi & Liu, Jinyuan & Wang, Shouxi & Qiao, Weibiao & Zeng, Fanhua, 2024. "Short-term load forecasting based on WM algorithm and transfer learning model," Applied Energy, Elsevier, vol. 353(PA).
    5. Kaiyan Wang & Haodong Du & Jiao Wang & Rong Jia & Zhenyu Zong, 2023. "An Ensemble Deep Learning Model for Provincial Load Forecasting Based on Reduced Dimensional Clustering and Decomposition Strategies," Mathematics, MDPI, vol. 11(12), pages 1-20, June.
    6. Xin Zhou & Xin Meng & Zhenyu Li, 2024. "ANN-LSTM-A Water Consumption Prediction Based on Attention Mechanism Enhancement," Energies, MDPI, vol. 17(5), pages 1-16, February.
    7. Gomez, William & Wang, Fu-Kwun & Lo, Shih-Che, 2024. "A hybrid approach based machine learning models in electricity markets," Energy, Elsevier, vol. 289(C).
    8. Zhao, Geya & Xue, Minggao & Cheng, Li, 2023. "A new hybrid model for multi-step WTI futures price forecasting based on self-attention mechanism and spatial–temporal graph neural network," Resources Policy, Elsevier, vol. 85(PB).
    9. Renxi Gong & Xianglong Li, 2023. "A Short-Term Load Forecasting Model Based on Crisscross Grey Wolf Optimizer and Dual-Stage Attention Mechanism," Energies, MDPI, vol. 16(6), pages 1-24, March.
    10. Shi, Jian & Teh, Jiashen, 2024. "Load forecasting for regional integrated energy system based on complementary ensemble empirical mode decomposition and multi-model fusion," Applied Energy, Elsevier, vol. 353(PB).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zang, Haixiang & Xu, Ruiqi & Cheng, Lilin & Ding, Tao & Liu, Ling & Wei, Zhinong & Sun, Guoqiang, 2021. "Residential load forecasting based on LSTM fusing self-attention mechanism with pooling," Energy, Elsevier, vol. 229(C).
    2. Chitalia, Gopal & Pipattanasomporn, Manisa & Garg, Vishal & Rahman, Saifur, 2020. "Robust short-term electrical load forecasting framework for commercial buildings using deep recurrent neural networks," Applied Energy, Elsevier, vol. 278(C).
    3. Sharma, Abhishek & Jain, Sachin Kumar, 2022. "A novel seasonal segmentation approach for day-ahead load forecasting," Energy, Elsevier, vol. 257(C).
    4. Sungwoo Park & Jihoon Moon & Seungwon Jung & Seungmin Rho & Sung Wook Baik & Eenjun Hwang, 2020. "A Two-Stage Industrial Load Forecasting Scheme for Day-Ahead Combined Cooling, Heating and Power Scheduling," Energies, MDPI, vol. 13(2), pages 1-23, January.
    5. Zhang, Liang & Wen, Jin & Li, Yanfei & Chen, Jianli & Ye, Yunyang & Fu, Yangyang & Livingood, William, 2021. "A review of machine learning in building load prediction," Applied Energy, Elsevier, vol. 285(C).
    6. Mingping Liu & Xihao Sun & Qingnian Wang & Suhui Deng, 2022. "Short-Term Load Forecasting Using EMD with Feature Selection and TCN-Based Deep Learning Model," Energies, MDPI, vol. 15(19), pages 1-22, September.
    7. Wang, Jianguo & Han, Lincheng & Zhang, Xiuyu & Wang, Yingzhou & Zhang, Shude, 2023. "Electrical load forecasting based on variable T-distribution and dual attention mechanism," Energy, Elsevier, vol. 283(C).
    8. Guo, Yabin & Wang, Jiangyu & Chen, Huanxin & Li, Guannan & Liu, Jiangyan & Xu, Chengliang & Huang, Ronggeng & Huang, Yao, 2018. "Machine learning-based thermal response time ahead energy demand prediction for building heating systems," Applied Energy, Elsevier, vol. 221(C), pages 16-27.
    9. Filipe Rodrigues & Carlos Cardeira & João M. F. Calado & Rui Melicio, 2023. "Short-Term Load Forecasting of Electricity Demand for the Residential Sector Based on Modelling Techniques: A Systematic Review," Energies, MDPI, vol. 16(10), pages 1-26, May.
    10. Zheng, Zhuang & Chen, Hainan & Luo, Xiaowei, 2019. "A Kalman filter-based bottom-up approach for household short-term load forecast," Applied Energy, Elsevier, vol. 250(C), pages 882-894.
    11. Li, Lechen & Meinrenken, Christoph J. & Modi, Vijay & Culligan, Patricia J., 2021. "Short-term apartment-level load forecasting using a modified neural network with selected auto-regressive features," Applied Energy, Elsevier, vol. 287(C).
    12. Xiao, Xun & Mo, Huadong & Zhang, Yinan & Shan, Guangcun, 2022. "Meta-ANN – A dynamic artificial neural network refined by meta-learning for Short-Term Load Forecasting," Energy, Elsevier, vol. 246(C).
    13. Xiao, Liye & Shao, Wei & Liang, Tulu & Wang, Chen, 2016. "A combined model based on multiple seasonal patterns and modified firefly algorithm for electrical load forecasting," Applied Energy, Elsevier, vol. 167(C), pages 135-153.
    14. Ijaz Ul Haq & Amin Ullah & Samee Ullah Khan & Noman Khan & Mi Young Lee & Seungmin Rho & Sung Wook Baik, 2021. "Sequential Learning-Based Energy Consumption Prediction Model for Residential and Commercial Sectors," Mathematics, MDPI, vol. 9(6), pages 1-17, March.
    15. Ibrahim, Muhammad Sohail & Dong, Wei & Yang, Qiang, 2020. "Machine learning driven smart electric power systems: Current trends and new perspectives," Applied Energy, Elsevier, vol. 272(C).
    16. Ling, Jihong & Zhang, Bingyang & Dai, Na & Xing, Jincheng, 2023. "Coupling input feature construction methods and machine learning algorithms for hourly secondary supply temperature prediction," Energy, Elsevier, vol. 278(C).
    17. Lai, Wenzhe & Zhen, Zhao & Wang, Fei & Fu, Wenjie & Wang, Junlong & Zhang, Xudong & Ren, Hui, 2024. "Sub-region division based short-term regional distributed PV power forecasting method considering spatio-temporal correlations," Energy, Elsevier, vol. 288(C).
    18. Zhu, Jizhong & Dong, Hanjiang & Zheng, Weiye & Li, Shenglin & Huang, Yanting & Xi, Lei, 2022. "Review and prospect of data-driven techniques for load forecasting in integrated energy systems," Applied Energy, Elsevier, vol. 321(C).
    19. Lu, Shixiang & Xu, Qifa & Jiang, Cuixia & Liu, Yezheng & Kusiak, Andrew, 2022. "Probabilistic load forecasting with a non-crossing sparse-group Lasso-quantile regression deep neural network," Energy, Elsevier, vol. 242(C).
    20. Manoj Manivannan & Behzad Najafi & Fabio Rinaldi, 2017. "Machine Learning-Based Short-Term Prediction of Air-Conditioning Load through Smart Meter Analytics," Energies, MDPI, vol. 10(11), pages 1-17, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:327:y:2022:i:c:s0306261922013204. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.