IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v7y2014i7p4185-4198d37677.html
   My bibliography  Save this article

Refined Diebold-Mariano Test Methods for the Evaluation of Wind Power Forecasting Models

Author

Listed:
  • Hao Chen

    (School of Electrical Engineering, Southeast University, No.2 Sipailou, Nanjing 210096, China)

  • Qiulan Wan

    (School of Electrical Engineering, Southeast University, No.2 Sipailou, Nanjing 210096, China)

  • Yurong Wang

    (School of Electrical Engineering, Southeast University, No.2 Sipailou, Nanjing 210096, China)

Abstract

The scientific evaluation methodology for the forecast accuracy of wind power forecasting models is an important issue in the domain of wind power forecasting. However, traditional forecast evaluation criteria, such as Mean Squared Error (MSE) and Mean Absolute Error (MAE), have limitations in application to some degree. In this paper, a modern evaluation criterion, the Diebold-Mariano (DM) test, is introduced. The DM test can discriminate the significant differences of forecasting accuracy between different models based on the scheme of quantitative analysis. Furthermore, the augmented DM test with rolling windows approach is proposed to give a more strict forecasting evaluation. By extending the loss function to an asymmetric structure, the asymmetric DM test is proposed. Case study indicates that the evaluation criteria based on DM test can relieve the influence of random sample disturbance. Moreover, the proposed augmented DM test can provide more evidence when the cost of changing models is expensive, and the proposed asymmetric DM test can add in the asymmetric factor, and provide practical evaluation of wind power forecasting models. It is concluded that the two refined DM tests can provide reference to the comprehensive evaluation for wind power forecasting models.

Suggested Citation

  • Hao Chen & Qiulan Wan & Yurong Wang, 2014. "Refined Diebold-Mariano Test Methods for the Evaluation of Wind Power Forecasting Models," Energies, MDPI, vol. 7(7), pages 1-14, July.
    • Handle: RePEc:gam:jeners:v:7:y:2014:i:7:p:4185-4198:d:37677
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/7/7/4185/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/7/7/4185/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Zhang, Yao & Wang, Jianxue & Wang, Xifan, 2014. "Review on probabilistic forecasting of wind power generation," Renewable and Sustainable Energy Reviews, Elsevier, vol. 32(C), pages 255-270.
    2. Bollerslev, Tim, 1986. "Generalized autoregressive conditional heteroskedasticity," Journal of Econometrics, Elsevier, vol. 31(3), pages 307-327, April.
    3. Manzano-Agugliaro, F. & Alcayde, A. & Montoya, F.G. & Zapata-Sierra, A. & Gil, C., 2013. "Scientific production of renewable energies worldwide: An overview," Renewable and Sustainable Energy Reviews, Elsevier, vol. 18(C), pages 134-143.
    4. Liu, Hui & Tian, Hong-qi & Pan, Di-fu & Li, Yan-fei, 2013. "Forecasting models for wind speed using wavelet, wavelet packet, time series and Artificial Neural Networks," Applied Energy, Elsevier, vol. 107(C), pages 191-208.
    5. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    6. Liu, Heping & Erdem, Ergin & Shi, Jing, 2011. "Comprehensive evaluation of ARMA-GARCH(-M) approaches for modeling the mean and volatility of wind speed," Applied Energy, Elsevier, vol. 88(3), pages 724-732, March.
    7. Hernandez-Escobedo, Quetzalcoatl & Manzano-Agugliaro, Francisco & Gazquez-Parra, Jose Antonio & Zapata-Sierra, Antonio, 2011. "Is the wind a periodical phenomenon? The case of Mexico," Renewable and Sustainable Energy Reviews, Elsevier, vol. 15(1), pages 721-728, January.
    8. Hernández-Escobedo, Q. & Manzano-Agugliaro, F. & Zapata-Sierra, A., 2010. "The wind power of Mexico," Renewable and Sustainable Energy Reviews, Elsevier, vol. 14(9), pages 2830-2840, December.
    9. De Giorgi, Maria Grazia & Ficarella, Antonio & Tarantino, Marco, 2011. "Error analysis of short term wind power prediction models," Applied Energy, Elsevier, vol. 88(4), pages 1298-1311, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. B. Shravan Kumar & Vadlamani Ravi & Rishabh Miglani, 2019. "Predicting Indian stock market using the psycho-linguistic features of financial news," Papers 1911.06193, arXiv.org.
    2. Haider A. Khan & Shahryar Ghorbani & Elham Shabani & Shahab S. Band, 2024. "Enhancement of Neural Networks Model’s Predictions of Currencies Exchange Rates by Phase Space Reconstruction and Harris Hawks’ Optimization," Computational Economics, Springer;Society for Computational Economics, vol. 63(2), pages 835-860, February.
    3. Xuejiao Ma & Dandan Liu, 2016. "Comparative Study of Hybrid Models Based on a Series of Optimization Algorithms and Their Application in Energy System Forecasting," Energies, MDPI, vol. 9(8), pages 1-34, August.
    4. Xiong, Xiaoping & Qing, Guohua, 2023. "A hybrid day-ahead electricity price forecasting framework based on time series," Energy, Elsevier, vol. 264(C).
    5. Theodore Syriopoulos & Michael Tsatsaronis & Ioannis Karamanos, 2021. "Support Vector Machine Algorithms: An Application to Ship Price Forecasting," Computational Economics, Springer;Society for Computational Economics, vol. 57(1), pages 55-87, January.
    6. Maya Malinda & Jo-Hui Chen, 2022. "The forecasting of consumer exchange-traded funds (ETFs) via grey relational analysis (GRA) and artificial neural network (ANN)," Empirical Economics, Springer, vol. 62(2), pages 779-823, February.
    7. Huang, Xiaoqiao & Li, Qiong & Tai, Yonghang & Chen, Zaiqing & Zhang, Jun & Shi, Junsheng & Gao, Bixuan & Liu, Wuming, 2021. "Hybrid deep neural model for hourly solar irradiance forecasting," Renewable Energy, Elsevier, vol. 171(C), pages 1041-1060.
    8. NICOLAE Simona & GRIGORE George-Eduard & MUȘETESCU Radu-Cristian, 2022. "The Use of GARCH Autoregressive Models in Estimating and Forecasting the Crude Oil Volatility," European Journal of Interdisciplinary Studies, Bucharest Economic Academy, issue 01, March.
    9. Li, Yuze & Jiang, Shangrong & Li, Xuerong & Wang, Shouyang, 2021. "The role of news sentiment in oil futures returns and volatility forecasting: Data-decomposition based deep learning approach," Energy Economics, Elsevier, vol. 95(C).
    10. B. Shravan Kumar & Vadlamani Ravi & Rishabh Miglani, 2021. "Predicting Indian Stock Market Using the Psycho-Linguistic Features of Financial News," Annals of Data Science, Springer, vol. 8(3), pages 517-558, September.
    11. Pourghorban, Mojtaba & Mamipour, Siab, 2020. "Modeling and Forecasting the Electricity Price in Iran Using Wavelet-Based GARCH Model," MPRA Paper 115042, University Library of Munich, Germany.
    12. Chin-Hung Kuan & Yungho Leu & Wen-Shin Lin & Chien-Pang Lee, 2022. "The Estimation of the Long-Term Agricultural Output with a Robust Machine Learning Prediction Model," Agriculture, MDPI, vol. 12(8), pages 1-15, July.
    13. Morais, Lucas Barros Scianni & Aquila, Giancarlo & de Faria, Victor Augusto Durães & Lima, Luana Medeiros Marangon & Lima, José Wanderley Marangon & de Queiroz, Anderson Rodrigo, 2023. "Short-term load forecasting using neural networks and global climate models: An application to a large-scale electrical power system," Applied Energy, Elsevier, vol. 348(C).
    14. Wu, Chunying & Wang, Jianzhou & Chen, Xuejun & Du, Pei & Yang, Wendong, 2020. "A novel hybrid system based on multi-objective optimization for wind speed forecasting," Renewable Energy, Elsevier, vol. 146(C), pages 149-165.
    15. Truong Ngoc Cuong & Sam-Sang You & Le Ngoc Bao Long & Hwan-Seong Kim, 2022. "Seaport Resilience Analysis and Throughput Forecast Using a Deep Learning Approach: A Case Study of Busan Port," Sustainability, MDPI, vol. 14(21), pages 1-25, October.
    16. Anh Ngoc-Lan Huynh & Ravinesh C. Deo & Duc-Anh An-Vo & Mumtaz Ali & Nawin Raj & Shahab Abdulla, 2020. "Near Real-Time Global Solar Radiation Forecasting at Multiple Time-Step Horizons Using the Long Short-Term Memory Network," Energies, MDPI, vol. 13(14), pages 1-30, July.
    17. Yongli Wang & Xiangyi Zhou & Hao Liu & Xichang Chen & Zixin Yan & Dexin Li & Chang Liu & Jiarui Wang, 2023. "Evaluation of the Maturity of Urban Energy Internet Development Based on AHP-Entropy Weight Method and Improved TOPSIS," Energies, MDPI, vol. 16(13), pages 1-18, July.
    18. Jun-Lin Lin & Yiqing Zhang & Kunhuang Zhu & Binbin Chen & Feng Zhang, 2020. "Asymmetric Loss Functions for Contract Capacity Optimization," Energies, MDPI, vol. 13(12), pages 1-13, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Shen, Zhiwei & Ritter, Matthias, 2016. "Forecasting volatility of wind power production," Applied Energy, Elsevier, vol. 176(C), pages 295-308.
    2. Hernández-Escobedo, Q. & Saldaña-Flores, R. & Rodríguez-García, E.R. & Manzano-Agugliaro, F., 2014. "Wind energy resource in Northern Mexico," Renewable and Sustainable Energy Reviews, Elsevier, vol. 32(C), pages 890-914.
    3. Ping Jiang & Zeng Wang & Kequan Zhang & Wendong Yang, 2017. "An Innovative Hybrid Model Based on Data Pre-Processing and Modified Optimization Algorithm and Its Application in Wind Speed Forecasting," Energies, MDPI, vol. 10(7), pages 1-29, July.
    4. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    5. Nima Amjady & Oveis Abedinia, 2017. "Short Term Wind Power Prediction Based on Improved Kriging Interpolation, Empirical Mode Decomposition, and Closed-Loop Forecasting Engine," Sustainability, MDPI, vol. 9(11), pages 1-18, November.
    6. Shahid, Farah & Zameer, Aneela & Mehmood, Ammara & Raja, Muhammad Asif Zahoor, 2020. "A novel wavenets long short term memory paradigm for wind power prediction," Applied Energy, Elsevier, vol. 269(C).
    7. Bentes, Sonia R. & Menezes, Rui, 2013. "On the predictability of realized volatility using feasible GLS," Journal of Asian Economics, Elsevier, vol. 28(C), pages 58-66.
    8. Dhanya Jothimani & Ravi Shankar & Surendra S. Yadav, 2016. "Discrete Wavelet Transform-Based Prediction of Stock Index: A Study on National Stock Exchange Fifty Index," Papers 1605.07278, arXiv.org.
    9. Wang, Jianzhou & Xiong, Shenghua, 2014. "A hybrid forecasting model based on outlier detection and fuzzy time series – A case study on Hainan wind farm of China," Energy, Elsevier, vol. 76(C), pages 526-541.
    10. Mohamed CHIKHI & Claude DIEBOLT, 2022. "Testing the weak form efficiency of the French ETF market with the LSTAR-ANLSTGARCH approach using a semiparametric estimation," Eastern Journal of European Studies, Centre for European Studies, Alexandru Ioan Cuza University, vol. 13, pages 228-253, June.
    11. Jun Lu & Shao Yi, 2022. "Reducing Overestimating and Underestimating Volatility via the Augmented Blending-ARCH Model," Applied Economics and Finance, Redfame publishing, vol. 9(2), pages 48-59, May.
    12. Fabrizio Cipollini & Robert F. Engle & Giampiero M. Gallo, 2016. "Copula--based Specification of vector MEMs," Papers 1604.01338, arXiv.org.
    13. Cruz-Peragon, F. & Palomar, J.M. & Casanova, P.J. & Dorado, M.P. & Manzano-Agugliaro, F., 2012. "Characterization of solar flat plate collectors," Renewable and Sustainable Energy Reviews, Elsevier, vol. 16(3), pages 1709-1720.
    14. Andrea Bucci, 2020. "Realized Volatility Forecasting with Neural Networks," Journal of Financial Econometrics, Oxford University Press, vol. 18(3), pages 502-531.
    15. Dimitrakopoulos, Stefanos & Tsionas, Mike, 2019. "Ordinal-response GARCH models for transaction data: A forecasting exercise," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1273-1287.
    16. Charles, Amélie, 2010. "The day-of-the-week effects on the volatility: The role of the asymmetry," European Journal of Operational Research, Elsevier, vol. 202(1), pages 143-152, April.
    17. Frank, Johannes, 2023. "Forecasting realized volatility in turbulent times using temporal fusion transformers," FAU Discussion Papers in Economics 03/2023, Friedrich-Alexander University Erlangen-Nuremberg, Institute for Economics.
    18. Mittnik, Stefan & Robinzonov, Nikolay & Spindler, Martin, 2015. "Stock market volatility: Identifying major drivers and the nature of their impact," Journal of Banking & Finance, Elsevier, vol. 58(C), pages 1-14.
    19. Dimitriadis, Timo & Schnaitmann, Julie, 2021. "Forecast encompassing tests for the expected shortfall," International Journal of Forecasting, Elsevier, vol. 37(2), pages 604-621.
    20. Peter Malec, 2016. "A Semiparametric Intraday GARCH Model," Cambridge Working Papers in Economics 1633, Faculty of Economics, University of Cambridge.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:7:y:2014:i:7:p:4185-4198:d:37677. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.