IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2609.27912.html

Global tree forecasters collapse at the hierarchical aggregate: a five-panel failure characterization

Author

Listed:
  • Md Rezwanul Islam
  • Wael Mohammed

Abstract

Global forecasting models pool many series and learn one shared function. Gradient-boosted trees are their most common form. We measure a failure of this design that has not, to our knowledge, been documented. Train a global tree on the individual series of a hierarchy, then ask it for the hierarchical aggregate. The aggregate sits far outside the model's training range, and the forecast collapses. The model under-predicts the total by 30-50x in our production deployment, and by up to 496x in a public M5 reconstruction. The mechanism is known: beyond its training range, a tree predicts a constant. It surfaces at the aggregate because the total dwarfs every training series. The cure is not new. Per-series scaling, the preprocessing step that Montero-Manso and Hyndman (2021) recommend, prevents the collapse. So do a weighted aggregate-level training row and seasonal differencing. Our contribution is the characterization. The collapse reproduces on five panels: a production business-to-business marketplace, a synthetic hierarchy, M5, Australian Tourism, and a public business-buyer panel. It holds on three tree libraries, is invariant across training seeds, and is statistically significant. Its onset is immediate and tracks a simple support bound: a scale gap of only 1.15x already costs a third of the total. No standard configuration change prevents it: pooling every hierarchy level into training fails at scale, and the one knob that fits linear models in the leaves softens it without curing it. Rolling the forecasts forward recursively separates the cures: the aggregate-row cure re-collapses, per-series scaling degrades but stays low, and only seasonal differencing keeps its one-step accuracy unchanged. We close with a three-step procedure for diagnosing and preventing the failure in deployed systems.

Suggested Citation

  • Md Rezwanul Islam & Wael Mohammed, 2026. "Global tree forecasters collapse at the hierarchical aggregate: a five-panel failure characterization," Papers 2609.27912, arXiv.org.
  • Handle: RePEc:arx:papers:2609.27912
    as

    Download full text from publisher

    File URL: https://arxiv.org/pdf/2609.27912
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ma, Shaohui & Fildes, Robert, 2022. "The performance of the global bottom-up approach in the M5 accuracy competition: A robustness check," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1492-1499.
    2. Montero-Manso, Pablo & Hyndman, Rob J., 2021. "Principles and algorithms for forecasting groups of time series: Locality and globality," International Journal of Forecasting, Elsevier, vol. 37(4), pages 1632-1653.
    3. Harvey, David & Leybourne, Stephen & Newbold, Paul, 1997. "Testing the equality of prediction mean squared errors," International Journal of Forecasting, Elsevier, vol. 13(2), pages 281-291, June.
    4. Hyndman, Rob J. & Koehler, Anne B., 2006. "Another look at measures of forecast accuracy," International Journal of Forecasting, Elsevier, vol. 22(4), pages 679-688.
    5. Sprangers, Olivier & Wadman, Wander & Schelter, Sebastian & de Rijke, Maarten, 2024. "Hierarchical forecasting at scale," International Journal of Forecasting, Elsevier, vol. 40(4), pages 1689-1700.
    6. Makridakis, Spyros & Spiliotis, Evangelos & Assimakopoulos, Vassilios, 2022. "M5 accuracy competition: Results, findings, and conclusions," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1346-1364.
    7. Salinas, David & Flunkert, Valentin & Gasthaus, Jan & Januschowski, Tim, 2020. "DeepAR: Probabilistic forecasting with autoregressive recurrent networks," International Journal of Forecasting, Elsevier, vol. 36(3), pages 1181-1191.
    8. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    9. Shanika L. Wickramasuriya & George Athanasopoulos & Rob J. Hyndman, 2019. "Optimal Forecast Reconciliation for Hierarchical and Grouped Time Series Through Trace Minimization," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 114(526), pages 804-819, April.
    10. Marcellino, Massimiliano & Stock, James H. & Watson, Mark W., 2006. "A comparison of direct and iterated multistep AR methods for forecasting macroeconomic time series," Journal of Econometrics, Elsevier, vol. 135(1-2), pages 499-526.
    11. Hyndman, Rob J. & Ahmed, Roman A. & Athanasopoulos, George & Shang, Han Lin, 2011. "Optimal combination forecasts for hierarchical time series," Computational Statistics & Data Analysis, Elsevier, vol. 55(9), pages 2579-2589, September.
    12. Anderer, Matthias & Li, Feng, 2022. "Hierarchical forecasting with a top-down alignment of independent-level forecasts," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1405-1414.
    13. Lainder, A. David & Wolfinger, Russell D., 2022. "Forecasting with gradient boosted trees: augmentation, tuning, and cross-validation strategies," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1426-1433.
    14. Januschowski, Tim & Wang, Yuyang & Torkkola, Kari & Erkkilä, Timo & Hasson, Hilaf & Gasthaus, Jan, 2022. "Forecasting with trees," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1473-1481.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Long, Xueying & Bui, Quang & Oktavian, Grady & Schmidt, Daniel F. & Bergmeir, Christoph & Godahewa, Rakshitha & Lee, Seong Per & Zhao, Kaifeng & Condylis, Paul, 2025. "Scalable probabilistic forecasting in retail with gradient boosted trees: A practitioner’s approach," International Journal of Production Economics, Elsevier, vol. 279(C).
    2. Rombouts, Jeroen & Ternes, Marie & Wilms, Ines, 2025. "Cross-temporal forecast reconciliation at digital platforms with machine learning," International Journal of Forecasting, Elsevier, vol. 41(1), pages 321-344.
    3. Marco Zanotti, 2025. "Do global forecasting models require frequent retraining?," Working Papers 551, University of Milano-Bicocca, Department of Economics.
    4. Eckert, Florian & Hyndman, Rob J. & Panagiotelis, Anastasios, 2021. "Forecasting Swiss exports using Bayesian forecast reconciliation," European Journal of Operational Research, Elsevier, vol. 291(2), pages 693-710.
    5. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    6. Sonnleitner, Benedikt & Kourentzes, Nikolaos & Ehrig, Claudia & Pflaum, Alexander, 2025. "Forecasting for optimization in road freight transport: A review," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 204(C).
    7. Sprangers, Olivier & Wadman, Wander & Schelter, Sebastian & de Rijke, Maarten, 2024. "Hierarchical forecasting at scale," International Journal of Forecasting, Elsevier, vol. 40(4), pages 1689-1700.
    8. Wellens, Arnoud P. & Boute, Robert N. & Udenio, Maximiliano, 2024. "Simplifying tree-based methods for retail sales forecasting with explanatory variables," European Journal of Operational Research, Elsevier, vol. 314(2), pages 523-539.
    9. Kamal Sanguri, 2026. "Selecting Forecasting Methods via Literature Prominence and Accuracy Testing: Evidence From Container Throughput Forecasting," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 45(6), pages 3041-3057, September.
    10. Athanasopoulos, George & Hyndman, Rob J. & Kourentzes, Nikolaos & Panagiotelis, Anastasios, 2024. "Forecast reconciliation: A review," International Journal of Forecasting, Elsevier, vol. 40(2), pages 430-456.
    11. Kafa, Nadine & Babai, M. Zied & Klibi, Walid, 2025. "Forecasting mail flow: A hierarchical approach for enhanced societal wellbeing," International Journal of Forecasting, Elsevier, vol. 41(1), pages 51-65.
    12. Jeroen Rombouts & Marie Ternes & Ines Wilms, 2024. "Cross-Temporal Forecast Reconciliation at Digital Platforms with Machine Learning," Papers 2402.09033, arXiv.org, revised May 2024.
    13. Nikolay Robinzonov & Gerhard Tutz & Torsten Hothorn, 2012. "Boosting techniques for nonlinear time series models," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 96(1), pages 99-122, January.
    14. Olivares, Kin G. & Meetei, O. Nganba & Ma, Ruijun & Reddy, Rohan & Cao, Mengfei & Dicker, Lee, 2024. "Probabilistic hierarchical forecasting with deep Poisson mixtures," International Journal of Forecasting, Elsevier, vol. 40(2), pages 470-489.
    15. Makridakis, Spyros & Spiliotis, Evangelos & Assimakopoulos, Vassilios, 2022. "Predicting/hypothesizing the findings of the M5 competition," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1337-1345.
    16. Andrea Carriero & Todd E. Clark & Massimiliano Marcellino, 2015. "Realtime nowcasting with a Bayesian mixed frequency model with stochastic volatility," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 178(4), pages 837-862, October.
    17. Dima, Bogdan & Dima, Ştefana Maria & Ioan, Roxana, 2025. "The short-run impact of investor expectations’ past volatility on current predictions: The case of VIX," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 98(C).
    18. Ling Tang & Chengyuan Zhang & Tingfei Li & Ling Li, 2021. "A novel BEMD-based method for forecasting tourist volume with search engine data," Tourism Economics, , vol. 27(5), pages 1015-1038, August.
    19. Oikonomou, Konstantinos & Damigos, Dimitris & Dimitriou, Dimitrios, 2025. "Globality in the metal markets: Leveraging cross-learning to forecast aluminum and copper prices," Resources Policy, Elsevier, vol. 103(C).
    20. Wilms, Ines & Rombouts, Jeroen & Croux, Christophe, 2021. "Multivariate volatility forecasts for stock market indices," International Journal of Forecasting, Elsevier, vol. 37(2), pages 484-499.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2609.27912. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: https://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.