IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v392y2025ics0306261925006610.html
   My bibliography  Save this article

AutoPQ: Automating quantile estimation from point forecasts in the context of sustainability

Author

Listed:
  • Meisenbacher, Stefan
  • Phipps, Kaleb
  • Taubert, Oskar
  • Weiel, Marie
  • Götz, Markus
  • Mikut, Ralf
  • Hagenmeyer, Veit

Abstract

Optimizing smart grid operations relies on critical decision-making informed by uncertainty quantification, making probabilistic forecasting a vital tool. However, designing such forecasting models presents three key challenges: achieving accurate and unbiased uncertainty quantification, reducing the workload for data scientists during the design process, and minimizing the environmental impact of model training. In order to address these challenges, we introduce AutoPQ, a novel method that fully automates and optimizes probabilistic forecasting for smart grid applications. AutoPQ enhances forecast uncertainty quantification by generating high-quality quantile forecasts from an existing point forecast by using a conditional Invertible Neural Network (cINN). Furthermore, AutoPQ automates the selection of the optimal point forecasting method and fine-tunes hyperparameters, ensuring the best-possible model and configuration for each application. For flexible adaptation to various performance needs and available computing power, AutoPQ comes with a default and an advanced configuration, making it suitable for a wide range of smart grid applications. We demonstrate that AutoPQ surpasses state-of-the-art probabilistic forecasting methods while effectively limiting computational effort and hence environmental impact. Additionally and in the context of sustainability, AutoPQ provides full transparency regarding the electricity consumption required for performance improvements.

Suggested Citation

  • Meisenbacher, Stefan & Phipps, Kaleb & Taubert, Oskar & Weiel, Marie & Götz, Markus & Mikut, Ralf & Hagenmeyer, Veit, 2025. "AutoPQ: Automating quantile estimation from point forecasts in the context of sustainability," Applied Energy, Elsevier, vol. 392(C).
  • Handle: RePEc:eee:appene:v:392:y:2025:i:c:s0306261925006610
    DOI: 10.1016/j.apenergy.2025.125931
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261925006610
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.apenergy.2025.125931?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. González Ordiano, Jorge Ángel & Gröll, Lutz & Mikut, Ralf & Hagenmeyer, Veit, 2020. "Probabilistic energy forecasting using the nearest neighbors quantile filter and quantile regression," International Journal of Forecasting, Elsevier, vol. 36(2), pages 310-323.
    2. Wang, Yi & Gan, Dahua & Sun, Mingyang & Zhang, Ning & Lu, Zongxiang & Kang, Chongqing, 2019. "Probabilistic individual load forecasting using pinball loss guided LSTM," Applied Energy, Elsevier, vol. 235(C), pages 10-20.
    3. Hyndman, Rob J. & Khandakar, Yeasmin, 2008. "Automatic Time Series Forecasting: The forecast Package for R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 27(i03).
    4. Hyndman, Rob J. & Koehler, Anne B. & Snyder, Ralph D. & Grose, Simone, 2002. "A state space framework for automatic forecasting using exponential smoothing methods," International Journal of Forecasting, Elsevier, vol. 18(3), pages 439-454.
    5. Wiese, Frauke & Schlecht, Ingmar & Bunke, Wolf-Dieter & Gerbaulet, Clemens & Hirth, Lion & Jahn, Martin & Kunz, Friedrich & Lorenz, Casimir & Mühlenpfordt, Jonathan & Reimann, Juliane & Schill, Wolf-P, 2019. "Open Power System Data – Frictionless data for electricity system modelling," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 236, pages 401-409.
    6. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    7. Appino, Riccardo Remo & González Ordiano, Jorge Ángel & Mikut, Ralf & Faulwasser, Timm & Hagenmeyer, Veit, 2018. "On the use of probabilistic forecasts in scheduling of renewable energy sources coupled to storages," Applied Energy, Elsevier, vol. 210(C), pages 1207-1218.
    8. Victor Chernozhukov & Kaspar Wuthrich & Yinchu Zhu, 2019. "Distributional conformal prediction," Papers 1909.07889, arXiv.org, revised Aug 2021.
    9. Hu, Jiaxiang & Hu, Weihao & Cao, Di & Sun, Xinwu & Chen, Jianjun & Huang, Yuehui & Chen, Zhe & Blaabjerg, Frede, 2024. "Probabilistic net load forecasting based on transformer network and Gaussian process-enabled residual modeling learning method," Renewable Energy, Elsevier, vol. 225(C).
    10. Dumas, Jonathan & Wehenkel, Antoine & Lanaspeze, Damien & Cornélusse, Bertrand & Sutera, Antonio, 2022. "A deep generative model for probabilistic energy forecasting in power systems: normalizing flows," Applied Energy, Elsevier, vol. 305(C).
    11. Zhang, Wenjie & Quan, Hao & Srinivasan, Dipti, 2018. "Parallel and reliable probabilistic load forecasting via quantile regression forest and quantile determination," Energy, Elsevier, vol. 160(C), pages 810-819.
    12. Hu, Jianming & Luo, Qingxi & Tang, Jingwei & Heng, Jiani & Deng, Yuwen, 2022. "Conformalized temporal convolutional quantile regression networks for wind power interval forecasting," Energy, Elsevier, vol. 248(C).
    13. Salinas, David & Flunkert, Valentin & Gasthaus, Jan & Januschowski, Tim, 2020. "DeepAR: Probabilistic forecasting with autoregressive recurrent networks," International Journal of Forecasting, Elsevier, vol. 36(3), pages 1181-1191.
    14. Hong, Tao & Pinson, Pierre & Fan, Shu & Zareipour, Hamidreza & Troccoli, Alberto & Hyndman, Rob J., 2016. "Probabilistic energy forecasting: Global Energy Forecasting Competition 2014 and beyond," International Journal of Forecasting, Elsevier, vol. 32(3), pages 896-913.
    15. Huber, Julian & Dann, David & Weinhardt, Christof, 2020. "Probabilistic forecasts of time and energy flexibility in battery electric vehicle charging," Applied Energy, Elsevier, vol. 262(C).
    16. Tilmann Gneiting & Fadoua Balabdaoui & Adrian E. Raftery, 2007. "Probabilistic forecasts, calibration and sharpness," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 69(2), pages 243-268, April.
    17. Zhang, Yi & Cheng, Chuntian & Cao, Rui & Li, Gang & Shen, Jianjian & Wu, Xinyu, 2021. "Multivariate probabilistic forecasting and its performance’s impacts on long-term dispatch of hydro-wind hybrid systems," Applied Energy, Elsevier, vol. 283(C).
    18. Sprangers, Olivier & Schelter, Sebastian & de Rijke, Maarten, 2023. "Parameter-efficient deep probabilistic forecasting," International Journal of Forecasting, Elsevier, vol. 39(1), pages 332-345.
    19. Brusaferri, Alessandro & Matteucci, Matteo & Spinelli, Stefano & Vitali, Andrea, 2022. "Probabilistic electric load forecasting through Bayesian Mixture Density Networks," Applied Energy, Elsevier, vol. 309(C).
    20. Henni, Sarah & Becker, Jonas & Staudt, Philipp & vom Scheidt, Frederik & Weinhardt, Christof, 2022. "Industrial peak shaving with battery storage using a probabilistic forecasting approach: Economic evaluation of risk attitude," Applied Energy, Elsevier, vol. 327(C).
    21. Cramer, Eike & Witthaut, Dirk & Mitsos, Alexander & Dahmen, Manuel, 2023. "Multivariate probabilistic forecasting of intraday electricity prices using normalizing flows," Applied Energy, Elsevier, vol. 346(C).
    22. He, Yaoyao & Liu, Rui & Li, Haiyan & Wang, Shuo & Lu, Xiaofen, 2017. "Short-term power load probability density forecasting method using kernel-based support vector quantile regression and Copula theory," Applied Energy, Elsevier, vol. 185(P1), pages 254-266.
    23. Lim, Bryan & Arık, Sercan Ö. & Loeff, Nicolas & Pfister, Tomas, 2021. "Temporal Fusion Transformers for interpretable multi-horizon time series forecasting," International Journal of Forecasting, Elsevier, vol. 37(4), pages 1748-1764.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wang, Shengjie & Kang, Yanfei & Petropoulos, Fotios, 2024. "Combining probabilistic forecasts of intermittent demand," European Journal of Operational Research, Elsevier, vol. 315(3), pages 1038-1048.
    2. Wen, Honglin & Pinson, Pierre & Gu, Jie & Jin, Zhijian, 2024. "Wind energy forecasting with missing values within a fully conditional specification framework," International Journal of Forecasting, Elsevier, vol. 40(1), pages 77-95.
    3. Liu, Mingping & Wang, Jialong & Deng, Suhui & Zhong, Chunxiao & Wang, Yuhao, 2025. "Short-term load probabilistic forecasting based on non-equidistant monotone composite quantile regression and improved MICN," Energy, Elsevier, vol. 320(C).
    4. Spiliotis, Evangelos & Makridakis, Spyros & Kaltsounis, Anastasios & Assimakopoulos, Vassilios, 2021. "Product sales probabilistic forecasting: An empirical evaluation using the M5 competition data," International Journal of Production Economics, Elsevier, vol. 240(C).
    5. Spiliotis, Evangelos & Petropoulos, Fotios, 2024. "On the update frequency of univariate forecasting models," European Journal of Operational Research, Elsevier, vol. 314(1), pages 111-121.
    6. Pesantez, Jorge E. & Li, Binbin & Lee, Christopher & Zhao, Zhizhen & Butala, Mark & Stillwell, Ashlynn S., 2023. "A Comparison Study of Predictive Models for Electricity Demand in a Diverse Urban Environment," Energy, Elsevier, vol. 283(C).
    7. Lu, Shixiang & Xu, Qifa & Jiang, Cuixia & Liu, Yezheng & Kusiak, Andrew, 2022. "Probabilistic load forecasting with a non-crossing sparse-group Lasso-quantile regression deep neural network," Energy, Elsevier, vol. 242(C).
    8. Makridakis, Spyros & Spiliotis, Evangelos & Assimakopoulos, Vassilios & Chen, Zhi & Gaba, Anil & Tsetlin, Ilia & Winkler, Robert L., 2022. "The M5 uncertainty competition: Results, findings and conclusions," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1365-1385.
    9. Makridakis, Spyros & Hyndman, Rob J. & Petropoulos, Fotios, 2020. "Forecasting in social settings: The state of the art," International Journal of Forecasting, Elsevier, vol. 36(1), pages 15-28.
    10. González Ordiano, Jorge Ángel & Gröll, Lutz & Mikut, Ralf & Hagenmeyer, Veit, 2020. "Probabilistic energy forecasting using the nearest neighbors quantile filter and quantile regression," International Journal of Forecasting, Elsevier, vol. 36(2), pages 310-323.
    11. He, Yaoyao & Cao, Chaojin & Wang, Shuo & Fu, Hong, 2022. "Nonparametric probabilistic load forecasting based on quantile combination in electrical power systems," Applied Energy, Elsevier, vol. 322(C).
    12. Semenoglou, Artemios-Anargyros & Spiliotis, Evangelos & Makridakis, Spyros & Assimakopoulos, Vassilios, 2021. "Investigating the accuracy of cross-learning time series forecasting methods," International Journal of Forecasting, Elsevier, vol. 37(3), pages 1072-1084.
    13. Theodorou, Evangelos & Spiliotis, Evangelos & Assimakopoulos, Vassilios, 2025. "Forecast accuracy and inventory performance: Insights on their relationship from the M5 competition data," European Journal of Operational Research, Elsevier, vol. 322(2), pages 414-426.
    14. Marcjasz, Grzegorz & Narajewski, Michał & Weron, Rafał & Ziel, Florian, 2023. "Distributional neural networks for electricity price forecasting," Energy Economics, Elsevier, vol. 125(C).
    15. Zhang, Shu & Wang, Yi & Zhang, Yutian & Wang, Dan & Zhang, Ning, 2020. "Load probability density forecasting by transforming and combining quantile forecasts," Applied Energy, Elsevier, vol. 277(C).
    16. Park, Jungyeon & Alvarenga, Estêvão & Jeon, Jooyoung & Li, Ran & Petropoulos, Fotios & Kim, Hokyun & Ahn, Kwangwon, 2024. "Probabilistic forecast-based portfolio optimization of electricity demand at low aggregation levels," Applied Energy, Elsevier, vol. 353(PB).
    17. Wang, Xiaoqian & Kang, Yanfei & Hyndman, Rob J. & Li, Feng, 2023. "Distributed ARIMA models for ultra-long time series," International Journal of Forecasting, Elsevier, vol. 39(3), pages 1163-1184.
    18. Agakishiev, Ilyas & Härdle, Wolfgang Karl & Kopa, Milos & Kozmik, Karel & Petukhina, Alla, 2025. "Multivariate probabilistic forecasting of electricity prices with trading applications," Energy Economics, Elsevier, vol. 141(C).
    19. Botman, Lola & Lago, Jesus & Becker, Thijs & Vanthournout, Koen & Moor, Bart De, 2025. "A global probabilistic approach for short-term forecasting of individual households electricity consumption," Applied Energy, Elsevier, vol. 382(C).
    20. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:392:y:2025:i:c:s0306261925006610. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.