IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v15y2022i24p9657-d1008479.html
   My bibliography  Save this article

Evaluation Metrics for Wind Power Forecasts: A Comprehensive Review and Statistical Analysis of Errors

Author

Listed:
  • Paweł Piotrowski

    (Electrical Power Engineering Institute, Warsaw University of Technology, Koszykowa 75 Street, 00-662 Warsaw, Poland)

  • Inajara Rutyna

    (Electrical Power Engineering Institute, Warsaw University of Technology, Koszykowa 75 Street, 00-662 Warsaw, Poland
    Ideas NCBR Sp. z o.o, Nowogrodzka Street 47A, 00-695 Warsaw, Poland)

  • Dariusz Baczyński

    (Electrical Power Engineering Institute, Warsaw University of Technology, Koszykowa 75 Street, 00-662 Warsaw, Poland)

  • Marcin Kopyt

    (Electrical Power Engineering Institute, Warsaw University of Technology, Koszykowa 75 Street, 00-662 Warsaw, Poland)

Abstract

Power generation forecasts for wind farms, especially with a short-term horizon, have been extensively researched due to the growing share of wind farms in total power generation. Detailed forecasts are necessary for the optimization of power systems of various sizes. This review and analytical paper is largely focused on a statistical analysis of forecasting errors based on more than one hundred papers on wind generation forecasts. Factors affecting the magnitude of forecasting errors are presented and discussed. Normalized root mean squared error (nRMSE) and normalized mean absolute error (nMAE) have been selected as the main error metrics considered here. A new and unique error dispersion factor (EDF) is proposed, being the ratio of nRMSE to nMAE. The variability of EDF depending on selected factors (size of wind farm, forecasting horizons, and class of forecasting method) has been examined. This is unique and original research, a novelty in studies on errors of power generation forecasts in wind farms. In addition, extensive quantitative and qualitative analyses have been conducted to assess the magnitude of forecasting error depending on selected factors (such as forecasting horizon, wind farm size, and a class of the forecasting method). Based on these analyses and a review of more than one hundred papers, a unique set of recommendations on the preferred content of papers addressing wind farm generation forecasts has been developed. These recommendations would make it possible to conduct very precise benchmarking meta-analyses of forecasting studies described in research papers and to develop valuable general conclusions concerning the analyzed phenomena.

Suggested Citation

  • Paweł Piotrowski & Inajara Rutyna & Dariusz Baczyński & Marcin Kopyt, 2022. "Evaluation Metrics for Wind Power Forecasts: A Comprehensive Review and Statistical Analysis of Errors," Energies, MDPI, vol. 15(24), pages 1-38, December.
    • Handle: RePEc:gam:jeners:v:15:y:2022:i:24:p:9657-:d:1008479
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/15/24/9657/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/15/24/9657/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Paweł Piotrowski & Dariusz Baczyński & Marcin Kopyt & Tomasz Gulczyński, 2022. "Advanced Ensemble Methods Using Machine Learning and Deep Learning for One-Day-Ahead Forecasts of Electric Energy Production in Wind Farms," Energies, MDPI, vol. 15(4), pages 1-30, February.
    2. Hu, Huanling & Wang, Lin & Lv, Sheng-Xiang, 2020. "Forecasting energy consumption and wind power generation using deep echo state network," Renewable Energy, Elsevier, vol. 154(C), pages 598-613.
    3. Branko Kosovic & Sue Ellen Haupt & Daniel Adriaansen & Stefano Alessandrini & Gerry Wiener & Luca Delle Monache & Yubao Liu & Seth Linden & Tara Jensen & William Cheng & Marcia Politovich & Paul Prest, 2020. "A Comprehensive Wind Power Forecasting System Integrating Artificial Intelligence and Numerical Weather Prediction," Energies, MDPI, vol. 13(6), pages 1-16, March.
    4. Bingchun Liu & Shijie Zhao & Xiaogang Yu & Lei Zhang & Qingshan Wang, 2020. "A Novel Deep Learning Approach for Wind Power Forecasting Based on WD-LSTM Model," Energies, MDPI, vol. 13(18), pages 1-17, September.
    5. Wang, Kejun & Qi, Xiaoxia & Liu, Hongda & Song, Jiakang, 2018. "Deep belief network based k-means cluster approach for short-term wind power forecasting," Energy, Elsevier, vol. 165(PA), pages 840-852.
    6. Niu, Zhewen & Yu, Zeyuan & Tang, Wenhu & Wu, Qinghua & Reformat, Marek, 2020. "Wind power forecasting using attention-based gated recurrent unit network," Energy, Elsevier, vol. 196(C).
    7. Dinh Thanh Viet & Vo Van Phuong & Minh Quan Duong & Quoc Tuan Tran, 2020. "Models for Short-Term Wind Power Forecasting Based on Improved Artificial Neural Network Using Particle Swarm Optimization and Genetic Algorithms," Energies, MDPI, vol. 13(11), pages 1-22, June.
    8. Lorenzo Donadio & Jiannong Fang & Fernando Porté-Agel, 2021. "Numerical Weather Prediction and Artificial Neural Network Coupling for Wind Energy Forecast," Energies, MDPI, vol. 14(2), pages 1-17, January.
    9. Yang, Hufang & Jiang, Ping & Wang, Ying & Li, Hongmin, 2022. "A fuzzy intelligent forecasting system based on combined fuzzification strategy and improved optimization algorithm for renewable energy power generation," Applied Energy, Elsevier, vol. 325(C).
    10. Li, Ranran & Jin, Yu, 2018. "A wind speed interval prediction system based on multi-objective optimization for machine learning method," Applied Energy, Elsevier, vol. 228(C), pages 2207-2220.
    11. Guangyu Qin & Qingyou Yan & Jingyao Zhu & Chuanbo Xu & Daniel M. Kammen, 2021. "Day-Ahead Wind Power Forecasting Based on Wind Load Data Using Hybrid Optimization Algorithm," Sustainability, MDPI, vol. 13(3), pages 1-17, January.
    12. Tawn, R. & Browell, J., 2022. "A review of very short-term wind and solar power forecasting," Renewable and Sustainable Energy Reviews, Elsevier, vol. 153(C).
    13. Hyndman, Rob J. & Koehler, Anne B., 2006. "Another look at measures of forecast accuracy," International Journal of Forecasting, Elsevier, vol. 22(4), pages 679-688.
    14. Paweł Piotrowski & Mirosław Parol & Piotr Kapler & Bartosz Fetliński, 2022. "Advanced Forecasting Methods of 5-Minute Power Generation in a PV System for Microgrid Operation Control," Energies, MDPI, vol. 15(7), pages 1-23, April.
    15. Lin, Zi & Liu, Xiaolei, 2020. "Wind power forecasting of an offshore wind turbine based on high-frequency SCADA data and deep learning neural network," Energy, Elsevier, vol. 201(C).
    16. Hong, Ying-Yi & Rioflorido, Christian Lian Paulo P., 2019. "A hybrid deep learning-based neural network for 24-h ahead wind power forecasting," Applied Energy, Elsevier, vol. 250(C), pages 530-539.
    17. Wang, Yun & Zou, Runmin & Liu, Fang & Zhang, Lingjun & Liu, Qianyi, 2021. "A review of wind speed and wind power forecasting with deep neural networks," Applied Energy, Elsevier, vol. 304(C).
    18. Paweł Piotrowski & Marcin Kopyt & Dariusz Baczyński & Sylwester Robak & Tomasz Gulczyński, 2021. "Hybrid and Ensemble Methods of Two Days Ahead Forecasts of Electric Energy Production in a Small Wind Turbine," Energies, MDPI, vol. 14(5), pages 1-25, February.
    19. Muhammad Shahzad Nazir & Fahad Alturise & Sami Alshmrany & Hafiz. M. J Nazir & Muhammad Bilal & Ahmad N. Abdalla & P. Sanjeevikumar & Ziad M. Ali, 2020. "Wind Generation Forecasting Methods and Proliferation of Artificial Neural Network: A Review of Five Years Research Trend," Sustainability, MDPI, vol. 12(9), pages 1-27, May.
    20. Sun, Mucun & Feng, Cong & Zhang, Jie, 2020. "Multi-distribution ensemble probabilistic wind power forecasting," Renewable Energy, Elsevier, vol. 148(C), pages 135-149.
    21. Wei Dong & Qiang Yang & Xinli Fang, 2018. "Multi-Step Ahead Wind Power Generation Prediction Based on Hybrid Machine Learning Techniques," Energies, MDPI, vol. 11(8), pages 1-19, July.
    22. Amirhosein Torabi & Sayyed Ali Kiaian Mousavy & Vahideh Dashti & Mohammadhossein Saeedi & Nasser Yousefi, 2019. "A New Prediction Model Based on Cascade NN for Wind Power Prediction," Computational Economics, Springer;Society for Computational Economics, vol. 53(3), pages 1219-1243, March.
    23. Khosravi, A. & Machado, L. & Nunes, R.O., 2018. "Time-series prediction of wind speed using machine learning algorithms: A case study Osorio wind farm, Brazil," Applied Energy, Elsevier, vol. 224(C), pages 550-566.
    24. Yu, Ruiguo & Liu, Zhiqiang & Li, Xuewei & Lu, Wenhuan & Ma, Degang & Yu, Mei & Wang, Jianrong & Li, Bin, 2019. "Scene learning: Deep convolutional networks for wind power prediction by embedding turbines into grid space," Applied Energy, Elsevier, vol. 238(C), pages 249-257.
    25. Shahram Hanifi & Xiaolei Liu & Zi Lin & Saeid Lotfian, 2020. "A Critical Review of Wind Power Forecasting Methods—Past, Present and Future," Energies, MDPI, vol. 13(15), pages 1-24, July.
    26. Sharifian, Amir & Ghadi, M. Jabbari & Ghavidel, Sahand & Li, Li & Zhang, Jiangfeng, 2018. "A new method based on Type-2 fuzzy neural network for accurate wind power forecasting under uncertain data," Renewable Energy, Elsevier, vol. 120(C), pages 220-230.
    27. Shahid, Farah & Zameer, Aneela & Mehmood, Ammara & Raja, Muhammad Asif Zahoor, 2020. "A novel wavenets long short term memory paradigm for wind power prediction," Applied Energy, Elsevier, vol. 269(C).
    28. Hugo Tavares Vieira Gouveia & Ronaldo Ribeiro Barbosa De Aquino & Aida Araújo Ferreira, 2018. "Enhancing Short-Term Wind Power Forecasting through Multiresolution Analysis and Echo State Networks," Energies, MDPI, vol. 11(4), pages 1-19, April.
    29. Wang, Cong & Zhang, Hongli & Ma, Ping, 2020. "Wind power forecasting based on singular spectrum analysis and a new hybrid Laguerre neural network," Applied Energy, Elsevier, vol. 259(C).
    30. Han, Li & Jing, Huitian & Zhang, Rongchang & Gao, Zhiyu, 2019. "Wind power forecast based on improved Long Short Term Memory network," Energy, Elsevier, vol. 189(C).
    31. Sun, Gaiping & Jiang, Chuanwen & Cheng, Pan & Liu, Yangyang & Wang, Xu & Fu, Yang & He, Yang, 2018. "Short-term wind power forecasts by a synthetical similar time series data mining method," Renewable Energy, Elsevier, vol. 115(C), pages 575-584.
    32. Dongxiao Niu & Di Pu & Shuyu Dai, 2018. "Ultra-Short-Term Wind-Power Forecasting Based on the Weighted Random Forest Optimized by the Niche Immune Lion Algorithm," Energies, MDPI, vol. 11(5), pages 1-21, April.
    33. Saira Al-Zadjali & Ahmed Al Maashri & Amer Al-Hinai & Rashid Al Abri & Swaroop Gajare & Sultan Al Yahyai & Mostafa Bakhtvar, 2021. "A Fast and Accurate Wind Speed and Direction Nowcasting Model for Renewable Energy Management Systems," Energies, MDPI, vol. 14(23), pages 1-20, November.
    34. Nourani Esfetang, Naser & Kazemzadeh, Rasool, 2018. "A novel hybrid technique for prediction of electric power generation in wind farms based on WIPSO, neural network and wavelet transform," Energy, Elsevier, vol. 149(C), pages 662-674.
    35. Xiaoyu Shi & Xuewen Lei & Qiang Huang & Shengzhi Huang & Kun Ren & Yuanyuan Hu, 2018. "Hourly Day-Ahead Wind Power Prediction Using the Hybrid Model of Variational Model Decomposition and Long Short-Term Memory," Energies, MDPI, vol. 11(11), pages 1-20, November.
    36. Ning Li & Fuxing He & Wentao Ma, 2019. "Wind Power Prediction Based on Extreme Learning Machine with Kernel Mean p -Power Error Loss," Energies, MDPI, vol. 12(4), pages 1-19, February.
    37. Jianguo Zhou & Xuechao Yu & Baoling Jin, 2018. "Short-Term Wind Power Forecasting: A New Hybrid Model Combined Extreme-Point Symmetric Mode Decomposition, Extreme Learning Machine and Particle Swarm Optimization," Sustainability, MDPI, vol. 10(9), pages 1-18, September.
    38. Jianguo Zhou & Xiaolei Xu & Xuejing Huo & Yushuo Li, 2019. "Forecasting Models for Wind Power Using Extreme-Point Symmetric Mode Decomposition and Artificial Neural Networks," Sustainability, MDPI, vol. 11(3), pages 1-23, January.
    39. Shahram Hanifi & Saeid Lotfian & Hossein Zare-Behtash & Andrea Cammarano, 2022. "Offshore Wind Power Forecasting—A New Hyperparameter Optimisation Algorithm for Deep Learning Models," Energies, MDPI, vol. 15(19), pages 1-21, September.
    40. Erick López & Carlos Valle & Héctor Allende & Esteban Gil & Henrik Madsen, 2018. "Wind Power Forecasting Based on Echo State Networks and Long Short-Term Memory," Energies, MDPI, vol. 11(3), pages 1-22, February.
    41. Yin, Hao & Ou, Zuhong & Huang, Shengquan & Meng, Anbo, 2019. "A cascaded deep learning wind power prediction approach based on a two-layer of mode decomposition," Energy, Elsevier, vol. 189(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Luis O. Lara-Cerecedo & Jesús F. Hinojosa & Nun Pitalúa-Díaz & Yasuhiro Matsumoto & Alvaro González-Angeles, 2023. "Prediction of the Electricity Generation of a 60-kW Photovoltaic System with Intelligent Models ANFIS and Optimized ANFIS-PSO," Energies, MDPI, vol. 16(16), pages 1-26, August.
    2. Grzegorz Dudek & Paweł Piotrowski & Dariusz Baczyński, 2023. "Intelligent Forecasting and Optimization in Electrical Power Systems: Advances in Models and Applications," Energies, MDPI, vol. 16(7), pages 1-11, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Paweł Piotrowski & Dariusz Baczyński & Marcin Kopyt & Tomasz Gulczyński, 2022. "Advanced Ensemble Methods Using Machine Learning and Deep Learning for One-Day-Ahead Forecasts of Electric Energy Production in Wind Farms," Energies, MDPI, vol. 15(4), pages 1-30, February.
    2. Ma, Zhengjing & Mei, Gang, 2022. "A hybrid attention-based deep learning approach for wind power prediction," Applied Energy, Elsevier, vol. 323(C).
    3. Zucatelli, P.J. & Nascimento, E.G.S. & Santos, A.Á.B. & Arce, A.M.G. & Moreira, D.M., 2021. "An investigation on deep learning and wavelet transform to nowcast wind power and wind power ramp: A case study in Brazil and Uruguay," Energy, Elsevier, vol. 230(C).
    4. Manisha Sawant & Rupali Patil & Tanmay Shikhare & Shreyas Nagle & Sakshi Chavan & Shivang Negi & Neeraj Dhanraj Bokde, 2022. "A Selective Review on Recent Advancements in Long, Short and Ultra-Short-Term Wind Power Prediction," Energies, MDPI, vol. 15(21), pages 1-24, October.
    5. Nathan Oaks Farrar & Mohd Hasan Ali & Dipankar Dasgupta, 2023. "Artificial Intelligence and Machine Learning in Grid Connected Wind Turbine Control Systems: A Comprehensive Review," Energies, MDPI, vol. 16(3), pages 1-25, February.
    6. Wang, Yun & Zou, Runmin & Liu, Fang & Zhang, Lingjun & Liu, Qianyi, 2021. "A review of wind speed and wind power forecasting with deep neural networks," Applied Energy, Elsevier, vol. 304(C).
    7. da Silva, Ramon Gomes & Ribeiro, Matheus Henrique Dal Molin & Moreno, Sinvaldo Rodrigues & Mariani, Viviana Cocco & Coelho, Leandro dos Santos, 2021. "A novel decomposition-ensemble learning framework for multi-step ahead wind energy forecasting," Energy, Elsevier, vol. 216(C).
    8. Upma Singh & Mohammad Rizwan & Muhannad Alaraj & Ibrahim Alsaidan, 2021. "A Machine Learning-Based Gradient Boosting Regression Approach for Wind Power Production Forecasting: A Step towards Smart Grid Environments," Energies, MDPI, vol. 14(16), pages 1-21, August.
    9. Jingtao Huang & Gang Niu & Haiping Guan & Shuzhong Song, 2023. "Ultra-Short-Term Wind Power Prediction Based on LSTM with Loss Shrinkage Adam," Energies, MDPI, vol. 16(9), pages 1-13, April.
    10. Ahmad, Tanveer & Zhang, Dongdong, 2022. "A data-driven deep sequence-to-sequence long-short memory method along with a gated recurrent neural network for wind power forecasting," Energy, Elsevier, vol. 239(PB).
    11. Wang, Cong & Zhang, Hongli & Ma, Ping, 2020. "Wind power forecasting based on singular spectrum analysis and a new hybrid Laguerre neural network," Applied Energy, Elsevier, vol. 259(C).
    12. Duan, Jiandong & Wang, Peng & Ma, Wentao & Tian, Xuan & Fang, Shuai & Cheng, Yulin & Chang, Ying & Liu, Haofan, 2021. "Short-term wind power forecasting using the hybrid model of improved variational mode decomposition and Correntropy Long Short -term memory neural network," Energy, Elsevier, vol. 214(C).
    13. Liu, Lei & Liu, Jicheng & Ye, Yu & Liu, Hui & Chen, Kun & Li, Dong & Dong, Xue & Sun, Mingzhai, 2023. "Ultra-short-term wind power forecasting based on deep Bayesian model with uncertainty," Renewable Energy, Elsevier, vol. 205(C), pages 598-607.
    14. Yang, Mao & Wang, Da & Xu, Chuanyu & Dai, Bozhi & Ma, Miaomiao & Su, Xin, 2023. "Power transfer characteristics in fluctuation partition algorithm for wind speed and its application to wind power forecasting," Renewable Energy, Elsevier, vol. 211(C), pages 582-594.
    15. Natei Ermias Benti & Mesfin Diro Chaka & Addisu Gezahegn Semie, 2023. "Forecasting Renewable Energy Generation with Machine Learning and Deep Learning: Current Advances and Future Prospects," Sustainability, MDPI, vol. 15(9), pages 1-33, April.
    16. Ma, Yixiang & Yu, Lean & Zhang, Guoxing, 2022. "Short-term wind power forecasting with an intermittency-trait-driven methodology," Renewable Energy, Elsevier, vol. 198(C), pages 872-883.
    17. Guanjun Liu & Chao Wang & Hui Qin & Jialong Fu & Qin Shen, 2022. "A Novel Hybrid Machine Learning Model for Wind Speed Probabilistic Forecasting," Energies, MDPI, vol. 15(19), pages 1-16, September.
    18. Ana Lagos & Joaquín E. Caicedo & Gustavo Coria & Andrés Romero Quete & Maximiliano Martínez & Gastón Suvire & Jesús Riquelme, 2022. "State-of-the-Art Using Bibliometric Analysis of Wind-Speed and -Power Forecasting Methods Applied in Power Systems," Energies, MDPI, vol. 15(18), pages 1-40, September.
    19. Bingchun Liu & Shijie Zhao & Xiaogang Yu & Lei Zhang & Qingshan Wang, 2020. "A Novel Deep Learning Approach for Wind Power Forecasting Based on WD-LSTM Model," Energies, MDPI, vol. 13(18), pages 1-17, September.
    20. Guangyu Qin & Qingyou Yan & Jingyao Zhu & Chuanbo Xu & Daniel M. Kammen, 2021. "Day-Ahead Wind Power Forecasting Based on Wind Load Data Using Hybrid Optimization Algorithm," Sustainability, MDPI, vol. 13(3), pages 1-17, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:15:y:2022:i:24:p:9657-:d:1008479. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.