IDEAS home Printed from https://ideas.repec.org/a/eee/intfor/v39y2023i2p981-991.html
   My bibliography  Save this article

Calibration of deterministic NWP forecasts and its impact on verification

Author

Listed:
  • Mayer, Martin János
  • Yang, Dazhi

Abstract

Deterministic forecasts (as opposed to ensemble or probabilistic forecasts) issued by numerical weather prediction (NWP) models require post-processing. Such corrective procedure can be viewed as a form of calibration. It is well known that, based on different objective functions, e.g., minimizing the mean square error or the mean absolute error, the calibrated forecasts have different impacts on verification. In this regard, this paper investigates how a calibration directive can affect various aspects of forecast quality outlined in the Murphy–Winkler distribution-oriented verification framework. It is argued that the correlation coefficient is the best measure for the potential performance of NWP forecast verification when linear calibration is involved, because (1) it is not affected by the directive of linear calibration, (2) it can be used to compute the skill score of the linearly calibrated forecasts, and (3) it can avoid the potential deficiency of using squared error to rank forecasts. Since no single error metric can fully represent all aspects of forecast quality, forecasters need to understand the trade-offs between different calibration strategies. To echo the increasing need to bridge atmospheric sciences, renewable energy engineering, and power system engineering, as to move toward the grand goal of carbon neutrality, this paper first provides a brief introduction to solar forecasting, and then revolves its discussion around a solar forecasting case study, such that the readers of this journal can gain further understanding on the subject and thus potentially contribute to it.

Suggested Citation

  • Mayer, Martin János & Yang, Dazhi, 2023. "Calibration of deterministic NWP forecasts and its impact on verification," International Journal of Forecasting, Elsevier, vol. 39(2), pages 981-991.
  • Handle: RePEc:eee:intfor:v:39:y:2023:i:2:p:981-991
    DOI: 10.1016/j.ijforecast.2022.03.008
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0169207022000486
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ijforecast.2022.03.008?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Peter Bauer & Alan Thorpe & Gilbert Brunet, 2015. "The quiet revolution of numerical weather prediction," Nature, Nature, vol. 525(7567), pages 47-55, September.
    2. Stephan Kolassa & Roland Martin, 2011. "Percentage Errors Can Ruin Your Day (and Rolling the Dice Shows How)," Foresight: The International Journal of Applied Forecasting, International Institute of Forecasters, issue 23, pages 21-27, Fall.
    3. Tao Hong & Pierre Pinson & Yi Wang & Rafal Weron & Dazhi Yang & Hamidreza Zareipour, 2020. "Energy forecasting: A review and outlook," WORking papers in Management Science (WORMS) WORMS/20/08, Department of Operations Research and Business Intelligence, Wroclaw University of Science and Technology.
    4. Yang, Dazhi & Wang, Wenting & Gueymard, Christian A. & Hong, Tao & Kleissl, Jan & Huang, Jing & Perez, Marc J. & Perez, Richard & Bright, Jamie M. & Xia, Xiang’ao & van der Meer, Dennis & Peters, Ian , 2022. "A review of solar forecasting, its dependence on atmospheric sciences and implications for grid integration: Towards carbon neutrality," Renewable and Sustainable Energy Reviews, Elsevier, vol. 161(C).
    5. Hong, Tao & Pinson, Pierre & Fan, Shu & Zareipour, Hamidreza & Troccoli, Alberto & Hyndman, Rob J., 2016. "Probabilistic energy forecasting: Global Energy Forecasting Competition 2014 and beyond," International Journal of Forecasting, Elsevier, vol. 32(3), pages 896-913.
    6. Makridakis, Spyros & Hyndman, Rob J. & Petropoulos, Fotios, 2020. "Forecasting in social settings: The state of the art," International Journal of Forecasting, Elsevier, vol. 36(1), pages 15-28.
    7. Mayer, Martin János & Gróf, Gyula, 2021. "Extensive comparison of physical models for photovoltaic power forecasting," Applied Energy, Elsevier, vol. 283(C).
    8. Yang, Dazhi & van der Meer, Dennis, 2021. "Post-processing in solar forecasting: Ten overarching thinking tools," Renewable and Sustainable Energy Reviews, Elsevier, vol. 140(C).
    9. Gneiting, Tilmann, 2011. "Making and Evaluating Point Forecasts," Journal of the American Statistical Association, American Statistical Association, vol. 106(494), pages 746-762.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yang, Dazhi & Gu, Yizhan & Mayer, Martin János & Gueymard, Christian A. & Wang, Wenting & Kleissl, Jan & Li, Mengying & Chu, Yinghao & Bright, Jamie M., 2024. "Regime-dependent 1-min irradiance separation model with climatology clustering," Renewable and Sustainable Energy Reviews, Elsevier, vol. 189(PA).
    2. Mayer, Martin János & Yang, Dazhi & Szintai, Balázs, 2023. "Comparing global and regional downscaled NWP models for irradiance and photovoltaic power forecasting: ECMWF versus AROME," Applied Energy, Elsevier, vol. 352(C).
    3. Sergiu-Mihai Hategan & Nicoleta Stefu & Marius Paulescu, 2023. "Calibration of GFS Solar Irradiation Forecasts: A Case Study in Romania," Energies, MDPI, vol. 16(11), pages 1-11, May.
    4. Chu, Yinghao & Yang, Dazhi & Yu, Hanxin & Zhao, Xin & Li, Mengying, 2024. "Can end-to-end data-driven models outperform traditional semi-physical models in separating 1-min irradiance?," Applied Energy, Elsevier, vol. 356(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yang, Dazhi & Kleissl, Jan, 2023. "Summarizing ensemble NWP forecasts for grid operators: Consistency, elicitability, and economic value," International Journal of Forecasting, Elsevier, vol. 39(4), pages 1640-1654.
    2. Mayer, Martin János, 2022. "Benefits of physical and machine learning hybridization for photovoltaic power forecasting," Renewable and Sustainable Energy Reviews, Elsevier, vol. 168(C).
    3. Mayer, Martin János & Yang, Dazhi, 2022. "Probabilistic photovoltaic power forecasting using a calibrated ensemble of model chains," Renewable and Sustainable Energy Reviews, Elsevier, vol. 168(C).
    4. Mayer, Martin János & Yang, Dazhi, 2023. "Pairing ensemble numerical weather prediction with ensemble physical model chain for probabilistic photovoltaic power forecasting," Renewable and Sustainable Energy Reviews, Elsevier, vol. 175(C).
    5. Markovics, Dávid & Mayer, Martin János, 2022. "Comparison of machine learning methods for photovoltaic power forecasting based on numerical weather prediction," Renewable and Sustainable Energy Reviews, Elsevier, vol. 161(C).
    6. Yang, Dazhi & Wang, Wenting & Gueymard, Christian A. & Hong, Tao & Kleissl, Jan & Huang, Jing & Perez, Marc J. & Perez, Richard & Bright, Jamie M. & Xia, Xiang’ao & van der Meer, Dennis & Peters, Ian , 2022. "A review of solar forecasting, its dependence on atmospheric sciences and implications for grid integration: Towards carbon neutrality," Renewable and Sustainable Energy Reviews, Elsevier, vol. 161(C).
    7. Zhang, Gang & Yang, Dazhi & Galanis, George & Androulakis, Emmanouil, 2022. "Solar forecasting with hourly updated numerical weather prediction," Renewable and Sustainable Energy Reviews, Elsevier, vol. 154(C).
    8. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    9. Gandhi, Oktoviano & Zhang, Wenjie & Kumar, Dhivya Sampath & Rodríguez-Gallegos, Carlos D. & Yagli, Gokhan Mert & Yang, Dazhi & Reindl, Thomas & Srinivasan, Dipti, 2024. "The value of solar forecasts and the cost of their errors: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 189(PB).
    10. Yang, Dazhi & van der Meer, Dennis, 2021. "Post-processing in solar forecasting: Ten overarching thinking tools," Renewable and Sustainable Energy Reviews, Elsevier, vol. 140(C).
    11. Wang, Wenting & Guo, Yufeng & Yang, Dazhi & Zhang, Zili & Kleissl, Jan & van der Meer, Dennis & Yang, Guoming & Hong, Tao & Liu, Bai & Huang, Nantian & Mayer, Martin János, 2024. "Economics of physics-based solar forecasting in power system day-ahead scheduling," Renewable and Sustainable Energy Reviews, Elsevier, vol. 199(C).
    12. Yang, Dazhi & Yang, Guoming & Liu, Bai, 2023. "Combining quantiles of calibrated solar forecasts from ensemble numerical weather prediction," Renewable Energy, Elsevier, vol. 215(C).
    13. Visser, L.R. & AlSkaif, T.A. & Khurram, A. & Kleissl, J. & van Sark, W.G.H.J.M., 2024. "Probabilistic solar power forecasting: An economic and technical evaluation of an optimal market bidding strategy," Applied Energy, Elsevier, vol. 370(C).
    14. Mayer, Martin János, 2022. "Impact of the tilt angle, inverter sizing factor and row spacing on the photovoltaic power forecast accuracy," Applied Energy, Elsevier, vol. 323(C).
    15. Liu, Bai & Yang, Dazhi & Mayer, Martin János & Coimbra, Carlos F.M. & Kleissl, Jan & Kay, Merlinde & Wang, Wenting & Bright, Jamie M. & Xia, Xiang’ao & Lv, Xin & Srinivasan, Dipti & Wu, Yan & Beyer, H, 2023. "Predictability and forecast skill of solar irradiance over the contiguous United States," Renewable and Sustainable Energy Reviews, Elsevier, vol. 182(C).
    16. Chen, Xiaoyang & Du, Yang & Lim, Enggee & Fang, Lurui & Yan, Ke, 2022. "Towards the applicability of solar nowcasting: A practice on predictive PV power ramp-rate control," Renewable Energy, Elsevier, vol. 195(C), pages 147-166.
    17. Yang, Dazhi, 2022. "Correlogram, predictability error growth, and bounds of mean square error of solar irradiance forecasts," Renewable and Sustainable Energy Reviews, Elsevier, vol. 167(C).
    18. Mayer, Martin János & Yang, Dazhi & Szintai, Balázs, 2023. "Comparing global and regional downscaled NWP models for irradiance and photovoltaic power forecasting: ECMWF versus AROME," Applied Energy, Elsevier, vol. 352(C).
    19. Wen, Honglin & Pinson, Pierre & Gu, Jie & Jin, Zhijian, 2024. "Wind energy forecasting with missing values within a fully conditional specification framework," International Journal of Forecasting, Elsevier, vol. 40(1), pages 77-95.
    20. Nikolaos Kolokas & Dimosthenis Ioannidis & Dimitrios Tzovaras, 2021. "Multi-Step Energy Demand and Generation Forecasting with Confidence Used for Specification-Free Aggregate Demand Optimization," Energies, MDPI, vol. 14(11), pages 1-36, May.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:intfor:v:39:y:2023:i:2:p:981-991. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/ijforecast .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.