IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v17y2024i23p6131-d1537393.html
   My bibliography  Save this article

New Forecasting Metrics Evaluated in Prophet, Random Forest, and Long Short-Term Memory Models for Load Forecasting

Author

Listed:
  • Prajowal Manandhar

    (DEWA R&D Centre, Dubai Electricity and Water Authority, Dubai P.O. Box 564, United Arab Emirates)

  • Hasan Rafiq

    (DEWA R&D Centre, Dubai Electricity and Water Authority, Dubai P.O. Box 564, United Arab Emirates)

  • Edwin Rodriguez-Ubinas

    (DEWA R&D Centre, Dubai Electricity and Water Authority, Dubai P.O. Box 564, United Arab Emirates)

  • Themis Palpanas

    (LIPADE, Université de Paris, 45 Rue Des Saints-Peres, 75006 Paris, France)

Abstract

Data mining is vital for smart grids because it enhances overall grid efficiency, enabling the analysis of large volumes of data, the optimization of energy distribution, the identification of patterns, and demand forecasting. Several performance metrics, such as the MAPE and RMSE, have been created to assess these forecasts. This paper presents new performance metrics called Evaluation Metrics for Performance Quantification (EMPQ), designed to evaluate forecasting models in a more comprehensive and detailed manner. These metrics fill the gap left by established metrics by assessing the likelihood of over- and under-forecasting. The proposed metrics quantify forecast bias through maximum and minimum deviation percentages, assessing the proximity of predicted values to actual consumption and differentiating between over- and under-forecasts. The effectiveness of these metrics is demonstrated through a comparative analysis of short-term load forecasting for residential customers in Dubai. This study was based on high-resolution smart meter data, weather data, and voluntary survey data of household characteristics, which permitted the subdivision of the customers into several groups. The new metrics were demonstrated on the Prophet, Random Forest (RF), and Long Short-term Memory (LSTM) models. EMPQ help to determine that the LSTM model exhibited a superior performance with a maximum deviation of approximately 10% for day-ahead and 20% for week-ahead forecasts in the “AC-included” category, outperforming the Prophet model, which had deviation rates of approximately 44% and 42%, respectively. EMPQ also help to determine that the RF excelled over LSTM for the ‘bedroom-number’ subcategory. The findings highlight the value of the proposed metrics in assessing model performance across diverse subcategories. This study demonstrates the value of tailored forecasting models for accurate load prediction and underscores the importance of enhanced performance metrics in informing model selection and supporting energy management strategies.

Suggested Citation

  • Prajowal Manandhar & Hasan Rafiq & Edwin Rodriguez-Ubinas & Themis Palpanas, 2024. "New Forecasting Metrics Evaluated in Prophet, Random Forest, and Long Short-Term Memory Models for Load Forecasting," Energies, MDPI, vol. 17(23), pages 1-30, December.
  • Handle: RePEc:gam:jeners:v:17:y:2024:i:23:p:6131-:d:1537393
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/17/23/6131/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/1996-1073/17/23/6131/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Sean J. Taylor & Benjamin Letham, 2018. "Forecasting at Scale," The American Statistician, Taylor & Francis Journals, vol. 72(1), pages 37-45, January.
    2. Prajowal Manandhar & Hasan Rafiq & Edwin Rodriguez-Ubinas & Juan David Barbosa & Omer Ahmed Qureshi & Mahmoud Tarek & Sgouris Sgouridis, 2022. "Understanding Energy Behavioral Changes Due to COVID-19 in the Residents of Dubai Using Electricity Consumption Data and Their Impacts," Energies, MDPI, vol. 16(1), pages 1-23, December.
    3. He, Feifei & Zhou, Jianzhong & Feng, Zhong-kai & Liu, Guangbiao & Yang, Yuqi, 2019. "A hybrid short-term load forecasting model based on variational mode decomposition and long short-term memory networks considering relevant factors with Bayesian optimization algorithm," Applied Energy, Elsevier, vol. 237(C), pages 103-116.
    4. Wan, Anping & Chang, Qing & AL-Bukhaiti, Khalil & He, Jiabo, 2023. "Short-term power load forecasting for combined heat and power using CNN-LSTM enhanced by attention mechanism," Energy, Elsevier, vol. 282(C).
    5. Sadaei, Hossein Javedani & de Lima e Silva, Petrônio Cândido & Guimarães, Frederico Gadelha & Lee, Muhammad Hisyam, 2019. "Short-term load forecasting by using a combined method of convolutional neural networks and fuzzy time series," Energy, Elsevier, vol. 175(C), pages 365-377.
    6. Wei, Nan & Yin, Lihua & Li, Chao & Wang, Wei & Qiao, Weibiao & Li, Changjun & Zeng, Fanhua & Fu, Lingdi, 2022. "Short-term load forecasting using detrend singular spectrum fluctuation analysis," Energy, Elsevier, vol. 256(C).
    7. Behm, Christian & Nolting, Lars & Praktiknjo, Aaron, 2020. "How to model European electricity load profiles using artificial neural networks," Applied Energy, Elsevier, vol. 277(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Vasileios Laitsos & Georgios Vontzos & Paschalis Paraschoudis & Eleftherios Tsampasis & Dimitrios Bargiotas & Lefteri H. Tsoukalas, 2024. "The State of the Art Electricity Load and Price Forecasting for the Modern Wholesale Electricity Market," Energies, MDPI, vol. 17(22), pages 1-37, November.
    2. Sharma, Abhishek & Jain, Sachin Kumar, 2022. "A novel seasonal segmentation approach for day-ahead load forecasting," Energy, Elsevier, vol. 257(C).
    3. Zang, Haixiang & Xu, Ruiqi & Cheng, Lilin & Ding, Tao & Liu, Ling & Wei, Zhinong & Sun, Guoqiang, 2021. "Residential load forecasting based on LSTM fusing self-attention mechanism with pooling," Energy, Elsevier, vol. 229(C).
    4. Thangjam, Aditya & Jaipuria, Sanjita & Dadabada, Pradeep Kumar, 2023. "Time-Varying approaches for Long-Term Electric Load Forecasting under economic shocks," Applied Energy, Elsevier, vol. 333(C).
    5. Mingping Liu & Xihao Sun & Qingnian Wang & Suhui Deng, 2022. "Short-Term Load Forecasting Using EMD with Feature Selection and TCN-Based Deep Learning Model," Energies, MDPI, vol. 15(19), pages 1-22, September.
    6. Bashiri Behmiri, Niaz & Fezzi, Carlo & Ravazzolo, Francesco, 2023. "Incorporating air temperature into mid-term electricity load forecasting models using time-series regressions and neural networks," Energy, Elsevier, vol. 278(C).
    7. Huiqun Yu & Haoyi Sun & Yueze Li & Chunmei Xu & Chenkun Du, 2024. "Enhanced Short-Term Load Forecasting: Error-Weighted and Hybrid Model Approach," Energies, MDPI, vol. 17(21), pages 1-22, October.
    8. Grzegorz Dudek, 2021. "Short-Term Load Forecasting Using Neural Networks with Pattern Similarity-Based Error Weights," Energies, MDPI, vol. 14(11), pages 1-18, May.
    9. Tavassoli-Hojati, Z. & Ghaderi, S.F. & Iranmanesh, H. & Hilber, P. & Shayesteh, E., 2020. "A self-partitioning local neuro fuzzy model for short-term load forecasting in smart grids," Energy, Elsevier, vol. 199(C).
    10. Fadaki, Masih & Asadikia, Atie, 2024. "Augmenting Monte Carlo Tree Search for managing service level agreements," International Journal of Production Economics, Elsevier, vol. 271(C).
    11. Miroslav Navratil & Andrea Kolkova, 2019. "Decomposition and Forecasting Time Series in the Business Economy Using Prophet Forecasting Model," Central European Business Review, Prague University of Economics and Business, vol. 2019(4), pages 26-39.
    12. Yang, Ting & Zhao, Liyuan & Li, Wei & Zomaya, Albert Y., 2021. "Dynamic energy dispatch strategy for integrated energy system based on improved deep reinforcement learning," Energy, Elsevier, vol. 235(C).
    13. Deng, Song & Dong, Xia & Tao, Li & Wang, Junjie & He, Yi & Yue, Dong, 2024. "Multi-type load forecasting model based on random forest and density clustering with the influence of noise and load patterns," Energy, Elsevier, vol. 307(C).
    14. Dong, Hanjiang & Zhu, Jizhong & Li, Shenglin & Wu, Wanli & Zhu, Haohao & Fan, Junwei, 2023. "Short-term residential household reactive power forecasting considering active power demand via deep Transformer sequence-to-sequence networks," Applied Energy, Elsevier, vol. 329(C).
    15. Ergun Yukseltan & Esra Agca Aktunc & Ayse H. Bilge & Ahmet Yucekaya, 2024. "An Overview of Electricity Consumption in Europe: Models for Prediction of the Electricity Usage for Heating and Cooling," International Journal of Energy Economics and Policy, Econjournals, vol. 14(2), pages 96-111, March.
    16. Zhewei Huang & Yawen Yi, 2024. "Short-Term Load Forecasting for Regional Smart Energy Systems Based on Two-Stage Feature Extraction and Hybrid Inverted Transformer," Sustainability, MDPI, vol. 16(17), pages 1-25, September.
    17. Ibrahim, Muhammad Sohail & Dong, Wei & Yang, Qiang, 2020. "Machine learning driven smart electric power systems: Current trends and new perspectives," Applied Energy, Elsevier, vol. 272(C).
    18. Leonard Burg & Gonca Gürses-Tran & Reinhard Madlener & Antonello Monti, 2021. "Comparative Analysis of Load Forecasting Models for Varying Time Horizons and Load Aggregation Levels," Energies, MDPI, vol. 14(21), pages 1-16, November.
    19. Sahar Koohfar & Wubeshet Woldemariam & Amit Kumar, 2023. "Performance Comparison of Deep Learning Approaches in Predicting EV Charging Demand," Sustainability, MDPI, vol. 15(5), pages 1-20, February.
    20. Md. Iftekharul Alam Efat & Petr Hajek & Mohammad Zoynul Abedin & Rahat Uddin Azad & Md. Al Jaber & Shuvra Aditya & Mohammad Kabir Hassan, 2024. "Deep-learning model using hybrid adaptive trend estimated series for modelling and forecasting sales," Annals of Operations Research, Springer, vol. 339(1), pages 297-328, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:17:y:2024:i:23:p:6131-:d:1537393. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.