IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2511.13616.html

Statistical and economic evaluation of forecasts in electricity markets: beyond RMSE and MAE

Author

Listed:
  • Katarzyna Maciejowska
  • Arkadiusz Lipiecki
  • Bartosz Uniejewski

Abstract

In recent years, a rapid development of forecasting methods has led to an increase in the accuracy of predictions. In the literature, forecasts are typically evaluated using metrics such as Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE). While appropriate for statistical assessment, these measures do not adequately reflect the economic value of forecasts. This study addresses the decision-making problem faced by a battery energy storage system, which must determine optimal charging and discharging times based on day-ahead electricity price forecasts. To explore the relationship between forecast accuracy and economic value, we generate a pool of 192 forecasts. These are evaluated using seven statistical metrics that go beyond RMSE and MAE, capturing various characteristics of the predictions and associated errors. We calculate the dynamic correlation between the statistical measures and gained profits to reveal that both RMSE and MAE are only weakly correlated with revenue. In contrast, measures that assess the alignment between predicted and actual daily price curves have a stronger relationship with profitability and are thus more effective for selecting optimal forecasts.

Suggested Citation

  • Katarzyna Maciejowska & Arkadiusz Lipiecki & Bartosz Uniejewski, 2025. "Statistical and economic evaluation of forecasts in electricity markets: beyond RMSE and MAE," Papers 2511.13616, arXiv.org.
    • Handle: RePEc:arx:papers:2511.13616
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2511.13616
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Marcjasz, Grzegorz & Uniejewski, Bartosz & Weron, Rafał, 2020. "Probabilistic electricity price forecasting with NARX networks: Combine point or probabilistic forecasts?," International Journal of Forecasting, Elsevier, vol. 36(2), pages 466-479.
    2. Lipiecki, Arkadiusz & Uniejewski, Bartosz & Weron, Rafał, 2024. "Postprocessing of point predictions for probabilistic forecasting of day-ahead electricity prices: The benefits of using isotonic distributional regression," Energy Economics, Elsevier, vol. 139(C).
    3. Amara-Ouali, Yvenn & Fasiolo, Matteo & Goude, Yannig & Yan, Hui, 2023. "Daily peak electrical load forecasting with a multi-resolution approach," International Journal of Forecasting, Elsevier, vol. 39(3), pages 1272-1286.
    4. Marcjasz, Grzegorz & Uniejewski, Bartosz & Weron, Rafał, 2019. "On the importance of the long-term seasonal component in day-ahead electricity price forecasting with NARX neural networks," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1520-1532.
    5. Maciejowska, Katarzyna & Nowotarski, Jakub & Weron, Rafał, 2016. "Probabilistic forecasting of electricity spot prices using Factor Quantile Regression Averaging," International Journal of Forecasting, Elsevier, vol. 32(3), pages 957-965.
    6. Gianfreda, Angelica & Grossi, Luigi, 2012. "Forecasting Italian electricity zonal prices with exogenous variables," Energy Economics, Elsevier, vol. 34(6), pages 2228-2239.
    7. Lindberg, O. & Zhu, R. & Widén, J., 2024. "Quantifying the value of probabilistic forecasts when trading renewable hybrid power parks in day-ahead markets: A Nordic case study," Renewable Energy, Elsevier, vol. 237(PB).
    8. Bartosz Uniejewski, 2024. "Regularization for electricity price forecasting," Operations Research and Decisions, Wroclaw University of Science and Technology, Faculty of Management, vol. 34(3), pages 267-286.
    9. Tomasz Serafin & Bartosz Uniejewski & Rafał Weron, 2019. "Averaging Predictive Distributions Across Calibration Windows for Day-Ahead Electricity Price Forecasting," Energies, MDPI, vol. 12(13), pages 1-12, July.
    10. Ziel, Florian & Weron, Rafał, 2018. "Day-ahead electricity price forecasting with high-dimensional structures: Univariate vs. multivariate modeling frameworks," Energy Economics, Elsevier, vol. 70(C), pages 396-420.
    11. Grzegorz Marcjasz & Tomasz Serafin & Rafał Weron, 2018. "Selection of Calibration Windows for Day-Ahead Electricity Price Forecasting," Energies, MDPI, vol. 11(9), pages 1-20, September.
    12. Marcjasz, Grzegorz & Narajewski, Michał & Weron, Rafał & Ziel, Florian, 2023. "Distributional neural networks for electricity price forecasting," Energy Economics, Elsevier, vol. 125(C).
    13. Janczura, Joanna & Trück, Stefan & Weron, Rafał & Wolff, Rodney C., 2013. "Identifying spikes and seasonal components in electricity spot price data: A guide to robust modeling," Energy Economics, Elsevier, vol. 38(C), pages 96-110.
    14. Lago, Jesus & Marcjasz, Grzegorz & De Schutter, Bart & Weron, Rafał, 2021. "Forecasting day-ahead electricity prices: A review of state-of-the-art algorithms, best practices and an open-access benchmark," Applied Energy, Elsevier, vol. 293(C).
    15. Serafin, Tomasz & Weron, Rafał, 2025. "Loss functions in regression models: Impact on profits and risk in day-ahead electricity trading," Energy Economics, Elsevier, vol. 148(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Chȩć, Katarzyna & Uniejewski, Bartosz & Weron, Rafał, 2025. "Extrapolating the long-term seasonal component of electricity prices for forecasting in the day-ahead market," Journal of Commodity Markets, Elsevier, vol. 37(C).
    2. Cerasa, Andrea & Zani, Alessandro, 2025. "Enhancing electricity price forecasting accuracy: A novel filtering strategy for improved out-of-sample predictions," Applied Energy, Elsevier, vol. 383(C).
    3. Katarzyna Maciejowska & Bartosz Uniejewski & Rafa{l} Weron, 2022. "Forecasting Electricity Prices," Papers 2204.11735, arXiv.org.
    4. Marcjasz, Grzegorz & Narajewski, Michał & Weron, Rafał & Ziel, Florian, 2023. "Distributional neural networks for electricity price forecasting," Energy Economics, Elsevier, vol. 125(C).
    5. Katarzyna Chk{e}'c & Bartosz Uniejewski & Rafa{l} Weron, 2025. "Extrapolating the long-term seasonal component of electricity prices for forecasting in the day-ahead market," Papers 2503.02518, arXiv.org.
    6. Uniejewski, Bartosz & Weron, Rafał, 2021. "Regularized quantile regression averaging for probabilistic electricity price forecasting," Energy Economics, Elsevier, vol. 95(C).
    7. Mira Watermeyer & Thomas Mobius & Oliver Grothe & Felix Musgens, 2023. "A hybrid model for day-ahead electricity price forecasting: Combining fundamental and stochastic modelling," Papers 2304.09336, arXiv.org.
    8. Sheybanivaziri, Samaneh & Le Dréau, Jérôme & Kazmi, Hussain, 2024. "Forecasting price spikes in day-ahead electricity markets: techniques, challenges, and the road ahead," Discussion Papers 2024/1, Norwegian School of Economics, Department of Business and Management Science.
    9. Michał Narajewski, 2022. "Probabilistic Forecasting of German Electricity Imbalance Prices," Energies, MDPI, vol. 15(14), pages 1-17, July.
    10. Hanus, Luboš & Baruník, Jozef, 2025. "Learning the probability distributions of day-ahead electricity prices," Energy Economics, Elsevier, vol. 152(C).
    11. Jonathan Berrisch & Florian Ziel, 2023. "Multivariate Probabilistic CRPS Learning with an Application to Day-Ahead Electricity Prices," Papers 2303.10019, arXiv.org, revised Feb 2024.
    12. Ciaran O'Connor & Mohamed Bahloul & Steven Prestwich & Andrea Visentin, 2025. "The Evolution of Probabilistic Price Forecasting Techniques: A Review of the Day-Ahead, Intra-Day, and Balancing Markets," Papers 2511.05523, arXiv.org.
    13. Berrisch, Jonathan & Ziel, Florian, 2024. "Multivariate probabilistic CRPS learning with an application to day-ahead electricity prices," International Journal of Forecasting, Elsevier, vol. 40(4), pages 1568-1586.
    14. Bartosz Uniejewski & Rafał Weron, 2018. "Efficient Forecasting of Electricity Spot Prices with Expert and LASSO Models," Energies, MDPI, vol. 11(8), pages 1-26, August.
    15. Grzegorz Marcjasz, 2020. "Forecasting Electricity Prices Using Deep Neural Networks: A Robust Hyper-Parameter Selection Scheme," Energies, MDPI, vol. 13(18), pages 1-18, September.
    16. Cornell, Cameron & Dinh, Nam Trong & Pourmousavi, S. Ali, 2024. "A probabilistic forecast methodology for volatile electricity prices in the Australian National Electricity Market," International Journal of Forecasting, Elsevier, vol. 40(4), pages 1421-1437.
    17. Özen, Kadir & Yıldırım, Dilem, 2021. "Application of bagging in day-ahead electricity price forecasting and factor augmentation," Energy Economics, Elsevier, vol. 103(C).
    18. Bartosz Uniejewski, 2023. "Smoothing Quantile Regression Averaging: A new approach to probabilistic forecasting of electricity prices," Papers 2302.00411, arXiv.org, revised Nov 2024.
    19. Lipiecki, Arkadiusz & Uniejewski, Bartosz & Weron, Rafał, 2024. "Postprocessing of point predictions for probabilistic forecasting of day-ahead electricity prices: The benefits of using isotonic distributional regression," Energy Economics, Elsevier, vol. 139(C).
    20. Serafin, Tomasz & Weron, Rafał, 2025. "Loss functions in regression models: Impact on profits and risk in day-ahead electricity trading," Energy Economics, Elsevier, vol. 148(C).

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2511.13616. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.