IDEAS home Printed from https://ideas.repec.org/a/eee/intfor/v38y2022i4p1325-1336.html
   My bibliography  Save this article

The M5 competition: Background, organization, and implementation

Author

Listed:
  • Makridakis, Spyros
  • Spiliotis, Evangelos
  • Assimakopoulos, Vassilios

Abstract

The M5 competition follows the previous four M competitions, whose purpose is to learn from empirical evidence how to improve forecasting performance and advance the theory and practice of forecasting. M5 focused on a retail sales forecasting application with the objective to produce the most accurate point forecasts for 42,840 time series that represent the hierarchical unit sales of the largest retail company in the world, Walmart, as well as to provide the most accurate estimates of the uncertainty of these forecasts. Hence, the competition consisted of two parallel challenges, namely the Accuracy and Uncertainty forecasting competitions. M5 extended the results of the previous M competitions by: (a) significantly expanding the number of participating methods, especially those in the category of machine learning; (b) evaluating the performance of the uncertainty distribution along with point forecast accuracy; (c) including exogenous/explanatory variables in addition to the time series data; (d) using grouped, correlated time series; and (e) focusing on series that display intermittency. This paper describes the background, organization, and implementations of the competition, and it presents the data used and their characteristics. Consequently, it serves as introductory material to the results of the two forecasting challenges to facilitate their understanding.

Suggested Citation

  • Makridakis, Spyros & Spiliotis, Evangelos & Assimakopoulos, Vassilios, 2022. "The M5 competition: Background, organization, and implementation," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1325-1336.
  • Handle: RePEc:eee:intfor:v:38:y:2022:i:4:p:1325-1336
    DOI: 10.1016/j.ijforecast.2021.07.007
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0169207021001187
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ijforecast.2021.07.007?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Athanasopoulos, George & Hyndman, Rob J., 2011. "The value of feedback in forecasting competitions," International Journal of Forecasting, Elsevier, vol. 27(3), pages 845-849.
    2. A A Syntetos & J E Boylan & J D Croston, 2005. "On the categorization of demand patterns," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 56(5), pages 495-503, May.
    3. Hong, Tao & Xie, Jingrui & Black, Jonathan, 2019. "Global energy forecasting competition 2017: Hierarchical probabilistic load forecasting," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1389-1399.
    4. Claeskens, Gerda & Magnus, Jan R. & Vasnev, Andrey L. & Wang, Wendun, 2016. "The forecast combination puzzle: A simple theoretical explanation," International Journal of Forecasting, Elsevier, vol. 32(3), pages 754-762.
    5. Boylan, John E. & Goodwin, Paul & Mohammadipour, Maryam & Syntetos, Aris A., 2015. "Reproducibility in forecasting research," International Journal of Forecasting, Elsevier, vol. 31(1), pages 79-90.
    6. Makridakis, Spyros & Chatfield, Chris & Hibon, Michele & Lawrence, Michael & Mills, Terence & Ord, Keith & Simmons, LeRoy F., 1993. "The M2-competition: A real-time judgmentally based forecasting study," International Journal of Forecasting, Elsevier, vol. 9(1), pages 5-22, April.
    7. Syntetos, Aris A. & Nikolopoulos, Konstantinos & Boylan, John E., 2010. "Judging the judges through accuracy-implication metrics: The case of inventory forecasting," International Journal of Forecasting, Elsevier, vol. 26(1), pages 134-143, January.
    8. Hyndman, Rob J., 2020. "A brief history of forecasting competitions," International Journal of Forecasting, Elsevier, vol. 36(1), pages 7-14.
    9. Huang, Tao & Fildes, Robert & Soopramanien, Didier, 2014. "The value of competitive information in forecasting FMCG retail product sales and the variable selection problem," European Journal of Operational Research, Elsevier, vol. 237(2), pages 738-748.
    10. Fildes, Robert & Ma, Shaohui & Kolassa, Stephan, 2019. "Retail forecasting: research and practice," MPRA Paper 89356, University Library of Munich, Germany.
    11. Syntetos, Aris A. & Boylan, John E., 2005. "The accuracy of intermittent demand estimates," International Journal of Forecasting, Elsevier, vol. 21(2), pages 303-314.
    12. Spiliotis, Evangelos & Kouloumos, Andreas & Assimakopoulos, Vassilios & Makridakis, Spyros, 2020. "Are forecasting competitions data representative of the reality?," International Journal of Forecasting, Elsevier, vol. 36(1), pages 37-53.
    13. Tashman, Leonard J., 2000. "Out-of-sample tests of forecasting accuracy: an analysis and review," International Journal of Forecasting, Elsevier, vol. 16(4), pages 437-450.
    14. Makridakis, Spyros & Spiliotis, Evangelos & Assimakopoulos, Vassilios, 2020. "The M4 Competition: 100,000 time series and 61 forecasting methods," International Journal of Forecasting, Elsevier, vol. 36(1), pages 54-74.
    15. Makridakis, Spyros & Hibon, Michele, 2000. "The M3-Competition: results, conclusions and implications," International Journal of Forecasting, Elsevier, vol. 16(4), pages 451-476.
    16. Ma, Shaohui & Fildes, Robert, 2017. "A retail store SKU promotions optimization model for category multi-period profit maximization," European Journal of Operational Research, Elsevier, vol. 260(2), pages 680-692.
    17. Petropoulos, Fotios & Hyndman, Rob J. & Bergmeir, Christoph, 2018. "Exploring the sources of uncertainty: Why does bagging for time series forecasting work?," European Journal of Operational Research, Elsevier, vol. 268(2), pages 545-554.
    18. Ma, Shaohui & Fildes, Robert & Huang, Tao, 2016. "Demand forecasting with high dimensional data: The case of SKU retail sales forecasting with intra- and inter-category promotional information," European Journal of Operational Research, Elsevier, vol. 249(1), pages 245-257.
    19. Spyros Makridakis & Evangelos Spiliotis & Vassilios Assimakopoulos, 2018. "Statistical and Machine Learning forecasting methods: Concerns and ways forward," PLOS ONE, Public Library of Science, vol. 13(3), pages 1-26, March.
    20. Alireza Pooya & Morteza Pakdaman & Lotfi Tadj, 2019. "Exact and approximate solution for optimal inventory control of two-stock with reworking and forecasting of demand," Operational Research, Springer, vol. 19(2), pages 333-346, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jeroen Rombouts & Marie Ternes & Ines Wilms, 2024. "Cross-Temporal Forecast Reconciliation at Digital Platforms with Machine Learning," Papers 2402.09033, arXiv.org.
    2. Makridakis, Spyros & Spiliotis, Evangelos & Assimakopoulos, Vassilios & Chen, Zhi & Gaba, Anil & Tsetlin, Ilia & Winkler, Robert L., 2022. "The M5 uncertainty competition: Results, findings and conclusions," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1365-1385.
    3. Prak, Dennis & Rogetzer, Patricia, 2022. "Timing intermittent demand with time-varying order-up-to levels," European Journal of Operational Research, Elsevier, vol. 303(3), pages 1126-1136.
    4. Makridakis, Spyros & Spiliotis, Evangelos & Assimakopoulos, Vassilios, 2022. "M5 accuracy competition: Results, findings, and conclusions," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1346-1364.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Makridakis, Spyros & Spiliotis, Evangelos & Assimakopoulos, Vassilios, 2022. "Predicting/hypothesizing the findings of the M5 competition," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1337-1345.
    2. Makridakis, Spyros & Spiliotis, Evangelos & Assimakopoulos, Vassilios, 2022. "M5 accuracy competition: Results, findings, and conclusions," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1346-1364.
    3. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    4. Makridakis, Spyros & Hyndman, Rob J. & Petropoulos, Fotios, 2020. "Forecasting in social settings: The state of the art," International Journal of Forecasting, Elsevier, vol. 36(1), pages 15-28.
    5. Makridakis, Spyros & Spiliotis, Evangelos & Assimakopoulos, Vassilios, 2020. "The M4 Competition: 100,000 time series and 61 forecasting methods," International Journal of Forecasting, Elsevier, vol. 36(1), pages 54-74.
    6. Makridakis, Spyros & Spiliotis, Evangelos & Assimakopoulos, Vassilios & Chen, Zhi & Gaba, Anil & Tsetlin, Ilia & Winkler, Robert L., 2022. "The M5 uncertainty competition: Results, findings and conclusions," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1365-1385.
    7. Kang, Yanfei & Spiliotis, Evangelos & Petropoulos, Fotios & Athiniotis, Nikolaos & Li, Feng & Assimakopoulos, Vassilios, 2021. "Déjà vu: A data-centric forecasting approach through time series cross-similarity," Journal of Business Research, Elsevier, vol. 132(C), pages 719-731.
    8. Evangelos Spiliotis & Spyros Makridakis & Artemios-Anargyros Semenoglou & Vassilios Assimakopoulos, 2022. "Comparison of statistical and machine learning methods for daily SKU demand forecasting," Operational Research, Springer, vol. 22(3), pages 3037-3061, July.
    9. Fotios Petropoulos & Evangelos Spiliotis, 2021. "The Wisdom of the Data: Getting the Most Out of Univariate Time Series Forecasting," Forecasting, MDPI, vol. 3(3), pages 1-20, June.
    10. Spiliotis, Evangelos & Makridakis, Spyros & Kaltsounis, Anastasios & Assimakopoulos, Vassilios, 2021. "Product sales probabilistic forecasting: An empirical evaluation using the M5 competition data," International Journal of Production Economics, Elsevier, vol. 240(C).
    11. Theodorou, Evangelos & Wang, Shengjie & Kang, Yanfei & Spiliotis, Evangelos & Makridakis, Spyros & Assimakopoulos, Vassilios, 2022. "Exploring the representativeness of the M5 competition data," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1500-1506.
    12. Fildes, Robert & Ma, Shaohui & Kolassa, Stephan, 2019. "Retail forecasting: research and practice," MPRA Paper 89356, University Library of Munich, Germany.
    13. Semenoglou, Artemios-Anargyros & Spiliotis, Evangelos & Makridakis, Spyros & Assimakopoulos, Vassilios, 2021. "Investigating the accuracy of cross-learning time series forecasting methods," International Journal of Forecasting, Elsevier, vol. 37(3), pages 1072-1084.
    14. Jennifer L. Castle & Jurgen A. Doornik & David F. Hendry, 2021. "Forecasting Principles from Experience with Forecasting Competitions," Forecasting, MDPI, vol. 3(1), pages 1-28, February.
    15. Wellens, Arnoud P. & Udenio, Maxi & Boute, Robert N., 2022. "Transfer learning for hierarchical forecasting: Reducing computational efforts of M5 winning methods," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1482-1491.
    16. Hyndman, Rob J., 2020. "A brief history of forecasting competitions," International Journal of Forecasting, Elsevier, vol. 36(1), pages 7-14.
    17. Spyros Makridakis & Chris Fry & Fotios Petropoulos & Evangelos Spiliotis, 2022. "The Future of Forecasting Competitions: Design Attributes and Principles," INFORMS Joural on Data Science, INFORMS, vol. 1(1), pages 96-113, April.
    18. Bojer, Casper Solheim & Meldgaard, Jens Peder, 2021. "Kaggle forecasting competitions: An overlooked learning opportunity," International Journal of Forecasting, Elsevier, vol. 37(2), pages 587-603.
    19. Fildes, Robert & Ma, Shaohui & Kolassa, Stephan, 2022. "Retail forecasting: Research and practice," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1283-1318.
    20. Spiliotis, Evangelos & Assimakopoulos, Vassilios & Makridakis, Spyros, 2020. "Generalizing the Theta method for automatic forecasting," European Journal of Operational Research, Elsevier, vol. 284(2), pages 550-558.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:intfor:v:38:y:2022:i:4:p:1325-1336. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/ijforecast .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.