IDEAS home Printed from https://ideas.repec.org/p/msh/ebswps/2020-26.html
   My bibliography  Save this paper

Probabilistic Forecast Reconciliation: Properties, Evaluation and Score Optimisation

Author

Listed:
  • Anastasios Panagiotelis
  • Puwasala Gamakumara
  • George Athanasopoulos
  • Rob J Hyndman

Abstract

We develop a framework for prediction of multivariate data that follow some known linear constraints, such as the example where some variables are aggregates of others. This is particularly common when forecasting time series (predicting the future), but also arises in other types of prediction. For point prediction, an increasingly popular technique is reconciliation, whereby predictions are made for all series (so-called `base' predictions) and subsequently adjusted to ensure coherence with the constraints. This paper extends reconciliation from the setting of point prediction to probabilistic prediction. A novel definition of reconciliation is developed and used to construct densities and draw samples from a reconciled probabilistic prediction. In the elliptical case, it is proven that the true predictive distribution can be recovered from reconciliation even when the location and scale matrix of the base prediction are chosen arbitrarily. To find reconciliation weights, an objective function based on scoring rules is optimised. The energy and variogram scores are considered since the log score is improper in the context of comparing unreconciled to reconciled predictions, a result also proved in this paper. To account for the stochastic nature of the energy and variogram scores, optimisation is achieved using stochastic gradient descent. This method is shown to improve base predictions in simulation studies and in an empirical application, particularly when the base prediction models are severely misspecified. When misspecification is not too severe, extending popular reconciliation methods for point prediction can result in a similar performance to score optimisation via stochastic gradient descent. The methods described here are implemented in the ProbReco package for R.

Suggested Citation

  • Anastasios Panagiotelis & Puwasala Gamakumara & George Athanasopoulos & Rob J Hyndman, 2020. "Probabilistic Forecast Reconciliation: Properties, Evaluation and Score Optimisation," Monash Econometrics and Business Statistics Working Papers 26/20, Monash University, Department of Econometrics and Business Statistics.
  • Handle: RePEc:msh:ebswps:2020-26
    as

    Download full text from publisher

    File URL: https://www.monash.edu/business/ebs/research/publications/ebs/wp26-2020.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Kourentzes, Nikolaos & Athanasopoulos, George, 2019. "Cross-temporal coherent forecasts for Australian tourism," Annals of Tourism Research, Elsevier, vol. 75(C), pages 393-409.
    2. Jeon, Jooyoung & Panagiotelis, Anastasios & Petropoulos, Fotios, 2019. "Probabilistic forecast reconciliation with applications to wind power and electric load," European Journal of Operational Research, Elsevier, vol. 279(2), pages 364-379.
    3. Eckert, Florian & Hyndman, Rob J. & Panagiotelis, Anastasios, 2021. "Forecasting Swiss exports using Bayesian forecast reconciliation," European Journal of Operational Research, Elsevier, vol. 291(2), pages 693-710.
    4. Shanika L. Wickramasuriya & George Athanasopoulos & Rob J. Hyndman, 2019. "Optimal Forecast Reconciliation for Hierarchical and Grouped Time Series Through Trace Minimization," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 114(526), pages 804-819, April.
    5. Shang, Han Lin & Haberman, Steven, 2017. "Grouped multivariate and functional time series forecasting:An application to annuity pricing," Insurance: Mathematics and Economics, Elsevier, vol. 75(C), pages 166-179.
    6. Panagiotelis, Anastasios & Athanasopoulos, George & Gamakumara, Puwasala & Hyndman, Rob J., 2021. "Forecast reconciliation: A geometric view with new insights on bias correction," International Journal of Forecasting, Elsevier, vol. 37(1), pages 343-359.
    7. Athanasopoulos, George & Ahmed, Roman A. & Hyndman, Rob J., 2009. "Hierarchical forecasts for Australian domestic tourism," International Journal of Forecasting, Elsevier, vol. 25(1), pages 146-166.
    8. Tim Janke & Florian Steinke, 2020. "Probabilistic multivariate electricity price forecasting using implicit generative ensemble post-processing," Papers 2005.13417, arXiv.org.
    9. Athanasopoulos, George & Hyndman, Rob J. & Kourentzes, Nikolaos & Petropoulos, Fotios, 2017. "Forecasting with temporal hierarchies," European Journal of Operational Research, Elsevier, vol. 262(1), pages 60-74.
    10. Schäfer Juliane & Strimmer Korbinian, 2005. "A Shrinkage Approach to Large-Scale Covariance Matrix Estimation and Implications for Functional Genomics," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 4(1), pages 1-32, November.
    11. Li, Han & Hyndman, Rob J., 2021. "Assessing mortality inequality in the U.S.: What can be said about the future?," Insurance: Mathematics and Economics, Elsevier, vol. 99(C), pages 152-162.
    12. Hyndman, Rob J. & Lee, Alan J. & Wang, Earo, 2016. "Fast computation of reconciled forecasts for hierarchical and grouped time series," Computational Statistics & Data Analysis, Elsevier, vol. 97(C), pages 16-32.
    13. Nystrup, Peter & Lindström, Erik & Pinson, Pierre & Madsen, Henrik, 2020. "Temporal hierarchies with autocorrelation for load forecasting," European Journal of Operational Research, Elsevier, vol. 280(3), pages 876-888.
    14. Kourentzes, Nikolaos & Athanasopoulos, George, 2021. "Elucidate structure in intermittent demand series," European Journal of Operational Research, Elsevier, vol. 288(1), pages 141-152.
    15. George Athanasopoulos & Nikolaos Kourentzes, 2020. "On the Evaluation of Hierarchical Forecasts," Monash Econometrics and Business Statistics Working Papers 2/20, Monash University, Department of Econometrics and Business Statistics.
    16. Yuan Gao & Han Lin Shang, 2017. "Multivariate Functional Time Series Forecasting: Application to Age-Specific Mortality Rates," Risks, MDPI, vol. 5(2), pages 1-18, March.
    17. Hyndman, Rob J. & Khandakar, Yeasmin, 2008. "Automatic Time Series Forecasting: The forecast Package for R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 27(i03).
    18. Hyndman, Rob J. & Ahmed, Roman A. & Athanasopoulos, George & Shang, Han Lin, 2011. "Optimal combination forecasts for hierarchical time series," Computational Statistics & Data Analysis, Elsevier, vol. 55(9), pages 2579-2589, September.
    19. Buhlmann P. & Yu B., 2003. "Boosting With the L2 Loss: Regression and Classification," Journal of the American Statistical Association, American Statistical Association, vol. 98, pages 324-339, January.
    20. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
    21. Carol Alexander & Michael Coulon & Yang Han & Xiaochun Meng, 2021. "Evaluating the Discrimination Ability of Proper Multivariate Scoring Rules," Papers 2101.12693, arXiv.org.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Chen, Zhi & Gaba, Anil & Tsetlin, Ilia & Winkler, Robert L., 2022. "Evaluating quantile forecasts in the M5 uncertainty competition," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1531-1545.
    2. Olivares, Kin G. & Meetei, O. Nganba & Ma, Ruijun & Reddy, Rohan & Cao, Mengfei & Dicker, Lee, 2024. "Probabilistic hierarchical forecasting with deep Poisson mixtures," International Journal of Forecasting, Elsevier, vol. 40(2), pages 470-489.
    3. Pritularga, Kandrika F. & Svetunkov, Ivan & Kourentzes, Nikolaos, 2021. "Stochastic coherency in forecast reconciliation," International Journal of Production Economics, Elsevier, vol. 240(C).
    4. Li, Han & Hyndman, Rob J., 2021. "Assessing mortality inequality in the U.S.: What can be said about the future?," Insurance: Mathematics and Economics, Elsevier, vol. 99(C), pages 152-162.
    5. Di Fonzo, Tommaso & Girolimetto, Daniele, 2024. "Forecast combination-based forecast reconciliation: Insights and extensions," International Journal of Forecasting, Elsevier, vol. 40(2), pages 490-514.
    6. Corani, Giorgio & Azzimonti, Dario & Rubattu, Nicolò, 2024. "Probabilistic reconciliation of count time series," International Journal of Forecasting, Elsevier, vol. 40(2), pages 457-469.
    7. Makridakis, Spyros & Spiliotis, Evangelos & Assimakopoulos, Vassilios & Chen, Zhi & Gaba, Anil & Tsetlin, Ilia & Winkler, Robert L., 2022. "The M5 uncertainty competition: Results, findings and conclusions," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1365-1385.
    8. Athanasopoulos, George & Hyndman, Rob J. & Kourentzes, Nikolaos & Panagiotelis, Anastasios, 2024. "Forecast reconciliation: A review," International Journal of Forecasting, Elsevier, vol. 40(2), pages 430-456.
    9. Katherine Tierney, 2022. "The Future of Assisted Reproductive Technology Live Births in the United States," Population Research and Policy Review, Springer;Southern Demographic Association (SDA), vol. 41(5), pages 2289-2309, October.
    10. Di Fonzo, Tommaso & Girolimetto, Daniele, 2023. "Cross-temporal forecast reconciliation: Optimal combination method and heuristic alternatives," International Journal of Forecasting, Elsevier, vol. 39(1), pages 39-57.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Athanasopoulos, George & Hyndman, Rob J. & Kourentzes, Nikolaos & Panagiotelis, Anastasios, 2024. "Forecast reconciliation: A review," International Journal of Forecasting, Elsevier, vol. 40(2), pages 430-456.
    2. Di Fonzo, Tommaso & Girolimetto, Daniele, 2023. "Cross-temporal forecast reconciliation: Optimal combination method and heuristic alternatives," International Journal of Forecasting, Elsevier, vol. 39(1), pages 39-57.
    3. Pritularga, Kandrika F. & Svetunkov, Ivan & Kourentzes, Nikolaos, 2021. "Stochastic coherency in forecast reconciliation," International Journal of Production Economics, Elsevier, vol. 240(C).
    4. Panagiotelis, Anastasios & Athanasopoulos, George & Gamakumara, Puwasala & Hyndman, Rob J., 2021. "Forecast reconciliation: A geometric view with new insights on bias correction," International Journal of Forecasting, Elsevier, vol. 37(1), pages 343-359.
    5. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    6. Hollyman, Ross & Petropoulos, Fotios & Tipping, Michael E., 2021. "Understanding forecast reconciliation," European Journal of Operational Research, Elsevier, vol. 294(1), pages 149-160.
    7. Di Fonzo, Tommaso & Girolimetto, Daniele, 2024. "Forecast combination-based forecast reconciliation: Insights and extensions," International Journal of Forecasting, Elsevier, vol. 40(2), pages 490-514.
    8. Athanasopoulos, George & Kourentzes, Nikolaos, 2023. "On the evaluation of hierarchical forecasts," International Journal of Forecasting, Elsevier, vol. 39(4), pages 1502-1511.
    9. Møller, Jan Kloppenborg & Nystrup, Peter & Madsen, Henrik, 2024. "Likelihood-based inference in temporal hierarchies," International Journal of Forecasting, Elsevier, vol. 40(2), pages 515-531.
    10. Nystrup, Peter & Lindström, Erik & Møller, Jan K. & Madsen, Henrik, 2021. "Dimensionality reduction in forecasting with temporal hierarchies," International Journal of Forecasting, Elsevier, vol. 37(3), pages 1127-1146.
    11. Zhang, Bohan & Kang, Yanfei & Panagiotelis, Anastasios & Li, Feng, 2023. "Optimal reconciliation with immutable forecasts," European Journal of Operational Research, Elsevier, vol. 308(2), pages 650-660.
    12. Yang, Yang & Shang, Han Lin & Raymer, James, 2024. "Forecasting Australian fertility by age, region, and birthplace," International Journal of Forecasting, Elsevier, vol. 40(2), pages 532-548.
    13. Li, Han & Chen, Hua, 2024. "Hierarchical mortality forecasting with EVT tails: An application to solvency capital requirement," International Journal of Forecasting, Elsevier, vol. 40(2), pages 549-563.
    14. Eckert, Florian & Hyndman, Rob J. & Panagiotelis, Anastasios, 2021. "Forecasting Swiss exports using Bayesian forecast reconciliation," European Journal of Operational Research, Elsevier, vol. 291(2), pages 693-710.
    15. Bergsteinsson, Hjörleifur G. & Møller, Jan Kloppenborg & Nystrup, Peter & Pálsson, Ólafur Pétur & Guericke, Daniela & Madsen, Henrik, 2021. "Heat load forecasting using adaptive temporal hierarchies," Applied Energy, Elsevier, vol. 292(C).
    16. Lila, Maurício Franca & Meira, Erick & Cyrino Oliveira, Fernando Luiz, 2022. "Forecasting unemployment in Brazil: A robust reconciliation approach using hierarchical data," Socio-Economic Planning Sciences, Elsevier, vol. 82(PB).
    17. George Athanasopoulos & Puwasala Gamakumara & Anastasios Panagiotelis & Rob J Hyndman & Mohamed Affan, 2019. "Hierarchical Forecasting," Monash Econometrics and Business Statistics Working Papers 2/19, Monash University, Department of Econometrics and Business Statistics.
    18. George Athanasopoulos & Nikolaos Kourentzes, 2021. "On the Evaluation of Hierarchical Forecasts," Monash Econometrics and Business Statistics Working Papers 10/21, Monash University, Department of Econometrics and Business Statistics.
    19. Leprince, Julien & Madsen, Henrik & Møller, Jan Kloppenborg & Zeiler, Wim, 2023. "Hierarchical learning, forecasting coherent spatio-temporal individual and aggregated building loads," Applied Energy, Elsevier, vol. 348(C).
    20. George Athanasopoulos & Nikolaos Kourentzes, 2020. "On the Evaluation of Hierarchical Forecasts," Monash Econometrics and Business Statistics Working Papers 2/20, Monash University, Department of Econometrics and Business Statistics.

    More about this item

    Keywords

    scoring rules; probabilistic forecasting; hierarchical time series; stochastic gradient descent;
    All these keywords.

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:msh:ebswps:2020-26. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Professor Xibin Zhang (email available below). General contact details of provider: https://edirc.repec.org/data/dxmonau.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.