IDEAS home Printed from https://ideas.repec.org/a/eee/intfor/v35y2019i1p251-265.html
   My bibliography  Save this article

The inventory performance of forecasting methods: Evidence from the M3 competition data

Author

Listed:
  • Petropoulos, Fotios
  • Wang, Xun
  • Disney, Stephen M.

Abstract

Forecasting competitions have been a major driver not only of improvements in forecasting methods’ performances, but also of the development of new forecasting approaches. However, despite the tremendous value and impact of these competitions, they do suffer from the limitation that performances are measured only in terms of the forecast accuracy and bias, ignoring utility metrics. Using the monthly industry series of the M3 competition, we empirically explore the inventory performances of various widely used forecasting techniques, including exponential smoothing, ARIMA models, the Theta method, and approaches based on multiple temporal aggregation. We employ a rolling simulation approach and analyse the results for the order-up-to policy under various lead times. We find that the methods that are based on combinations result in superior inventory performances, while the Naïve, Holt, and Holt-Winters methods perform poorly.

Suggested Citation

  • Petropoulos, Fotios & Wang, Xun & Disney, Stephen M., 2019. "The inventory performance of forecasting methods: Evidence from the M3 competition data," International Journal of Forecasting, Elsevier, vol. 35(1), pages 251-265.
  • Handle: RePEc:eee:intfor:v:35:y:2019:i:1:p:251-265
    DOI: 10.1016/j.ijforecast.2018.01.004
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0169207018300232
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ijforecast.2018.01.004?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. A A Syntetos & J E Boylan & J D Croston, 2005. "On the categorization of demand patterns," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 56(5), pages 495-503, May.
    2. Petropoulos, Fotios & Kourentzes, Nikolaos & Nikolopoulos, Konstantinos, 2016. "Another look at estimators for intermittent demand," International Journal of Production Economics, Elsevier, vol. 181(PA), pages 154-161.
    3. Hyndman, Rob J. & Khandakar, Yeasmin, 2008. "Automatic Time Series Forecasting: The forecast Package for R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 27(i03).
    4. Zied Babai, M. & Syntetos, Aris A. & Teunter, Ruud, 2010. "On the empirical performance of (T, s, S) heuristics," European Journal of Operational Research, Elsevier, vol. 202(2), pages 466-472, April.
    5. Hedenstierna, Carl Philip T. & Disney, Stephen M., 2016. "Inventory performance under staggered deliveries and autocorrelated demand," European Journal of Operational Research, Elsevier, vol. 249(3), pages 1082-1091.
    6. Tashman, Leonard J., 2000. "Out-of-sample tests of forecasting accuracy: an analysis and review," International Journal of Forecasting, Elsevier, vol. 16(4), pages 437-450.
    7. Alan S. Blinder & Louis J. Maccini, 1991. "Taking Stock: A Critical Assessment of Recent Research on Inventories," Journal of Economic Perspectives, American Economic Association, vol. 5(1), pages 73-96, Winter.
    8. Fiorucci, Jose A. & Pellegrini, Tiago R. & Louzada, Francisco & Petropoulos, Fotios & Koehler, Anne B., 2016. "Models for optimising the theta method and their relationship to state space models," International Journal of Forecasting, Elsevier, vol. 32(4), pages 1151-1161.
    9. Assimakopoulos, V. & Nikolopoulos, K., 2000. "The theta model: a decomposition approach to forecasting," International Journal of Forecasting, Elsevier, vol. 16(4), pages 521-530.
    10. Everette S. Gardner, 1990. "Evaluating Forecast Performance in an Inventory Control System," Management Science, INFORMS, vol. 36(4), pages 490-499, April.
    11. Silver, Edward A. & Robb, David J., 2008. "Some insights regarding the optimal reorder period in periodic review inventory systems," International Journal of Production Economics, Elsevier, vol. 112(1), pages 354-366, March.
    12. Xun Wang & Fotios Petropoulos, 2016. "To select or to combine? The inventory performance of model and expert forecasts," International Journal of Production Research, Taylor & Francis Journals, vol. 54(17), pages 5271-5282, September.
    13. Syntetos, Aris A. & Zied Babai, M. & Gardner, Everette S., 2015. "Forecasting intermittent inventory demands: simple parametric methods vs. bootstrapping," Journal of Business Research, Elsevier, vol. 68(8), pages 1746-1752.
    14. Boylan, John E. & Goodwin, Paul & Mohammadipour, Maryam & Syntetos, Aris A., 2015. "Reproducibility in forecasting research," International Journal of Forecasting, Elsevier, vol. 31(1), pages 79-90.
    15. Hyndman, Rob J. & Koehler, Anne B., 2006. "Another look at measures of forecast accuracy," International Journal of Forecasting, Elsevier, vol. 22(4), pages 679-688.
    16. Kourentzes, Nikolaos & Petropoulos, Fotios & Trapero, Juan R., 2014. "Improving forecasting by estimating time series structural components across multiple frequencies," International Journal of Forecasting, Elsevier, vol. 30(2), pages 291-302.
    17. Syntetos, Aris A. & Nikolopoulos, Konstantinos & Boylan, John E., 2010. "Judging the judges through accuracy-implication metrics: The case of inventory forecasting," International Journal of Forecasting, Elsevier, vol. 26(1), pages 134-143, January.
    18. Snyder, Ralph D. & Koehler, Anne B. & Ord, J. Keith, 2002. "Forecasting for inventory control with exponential smoothing," International Journal of Forecasting, Elsevier, vol. 18(1), pages 5-18.
    19. Makridakis, Spyros & Hibon, Michele, 2000. "The M3-Competition: results, conclusions and implications," International Journal of Forecasting, Elsevier, vol. 16(4), pages 451-476.
    20. Barrow, Devon K. & Kourentzes, Nikolaos, 2016. "Distributions of forecasting errors of forecast combinations: Implications for inventory management," International Journal of Production Economics, Elsevier, vol. 177(C), pages 24-33.
    21. Hyndman, Rob J. & Billah, Baki, 2003. "Unmasking the Theta method," International Journal of Forecasting, Elsevier, vol. 19(2), pages 287-290.
    22. Kourentzes, Nikolaos, 2013. "Intermittent demand forecasts with neural networks," International Journal of Production Economics, Elsevier, vol. 143(1), pages 198-206.
    23. Syntetos, Aris A. & Boylan, John E., 2005. "The accuracy of intermittent demand estimates," International Journal of Forecasting, Elsevier, vol. 21(2), pages 303-314.
    24. Gérard P. Cachon & Taylor Randall & Glen M. Schmidt, 2007. "In Search of the Bullwhip Effect," Manufacturing & Service Operations Management, INFORMS, vol. 9(4), pages 457-479, April.
    25. Syntetos, Aris A. & Nikolopoulos, Konstantinos & Boylan, John E. & Fildes, Robert & Goodwin, Paul, 2009. "The effects of integrating management judgement into intermittent demand forecasts," International Journal of Production Economics, Elsevier, vol. 118(1), pages 72-81, March.
    26. Kourentzes, Nikolaos, 2014. "On intermittent demand model optimisation and selection," International Journal of Production Economics, Elsevier, vol. 156(C), pages 180-190.
    27. Disney, S. M. & Towill, D. R. & van de Velde, W., 2004. "Variance amplification and the golden ratio in production and inventory control," International Journal of Production Economics, Elsevier, vol. 90(3), pages 295-309, August.
    28. Hau L. Lee & V. Padmanabhan & Seungjin Whang, 1997. "Information Distortion in a Supply Chain: The Bullwhip Effect," Management Science, INFORMS, vol. 43(4), pages 546-558, April.
    29. Disney, Stephen M. & Gaalman, Gerard J.C. & Hedenstierna, Carl Philip T. & Hosoda, Takamichi, 2015. "Fill rate in a periodic review order-up-to policy under auto-correlated normally distributed, possibly negative, demand," International Journal of Production Economics, Elsevier, vol. 170(PB), pages 501-512.
    30. Wang, Xun & Disney, Stephen M., 2016. "The bullwhip effect: Progress, trends and directions," European Journal of Operational Research, Elsevier, vol. 250(3), pages 691-701.
    31. Frank Chen & Zvi Drezner & Jennifer K. Ryan & David Simchi-Levi, 2000. "Quantifying the Bullwhip Effect in a Simple Supply Chain: The Impact of Forecasting, Lead Times, and Information," Management Science, INFORMS, vol. 46(3), pages 436-443, March.
    32. Hosoda, Takamichi & Disney, Stephen M., 2012. "A delayed demand supply chain: Incentives for upstream players," Omega, Elsevier, vol. 40(4), pages 478-487.
    33. J E Boylan & A A Syntetos & G C Karakostas, 2008. "Classification for forecasting and stock control: a case study," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 59(4), pages 473-481, April.
    34. Hosoda, Takamichi & Disney, Stephen M., 2006. "On variance amplification in a three-echelon supply chain with minimum mean square error forecasting," Omega, Elsevier, vol. 34(4), pages 344-358, August.
    35. Ali, Mohammad M. & Boylan, John E. & Syntetos, Aris A., 2012. "Forecast errors and inventory performance under forecast information sharing," International Journal of Forecasting, Elsevier, vol. 28(4), pages 830-841.
    36. Syntetos, Aris A. & Boylan, John E., 2006. "On the stock control performance of intermittent demand estimators," International Journal of Production Economics, Elsevier, vol. 103(1), pages 36-47, September.
    37. Strijbosch, Leo W.G. & Syntetos, Aris A. & Boylan, John E. & Janssen, Elleke, 2011. "On the interaction between forecasting and stock control: The case of non-stationary demand," International Journal of Production Economics, Elsevier, vol. 133(1), pages 470-480, September.
    38. R H Teunter & L Duncan, 2009. "Forecasting intermittent demand: a comparative study," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 60(3), pages 321-329, March.
    39. Koning, Alex J. & Franses, Philip Hans & Hibon, Michele & Stekler, H.O., 2005. "The M3 competition: Statistical tests of the results," International Journal of Forecasting, Elsevier, vol. 21(3), pages 397-409.
    40. Li, Qinyun & Disney, Stephen M. & Gaalman, Gerard, 2014. "Avoiding the bullwhip effect using Damped Trend forecasting and the Order-Up-To replenishment policy," International Journal of Production Economics, Elsevier, vol. 149(C), pages 3-16.
    41. Warren Liao, T. & Chang, P.C., 2010. "Impacts of forecast, inventory policy, and lead time on supply chain inventory--A numerical study," International Journal of Production Economics, Elsevier, vol. 128(2), pages 527-537, December.
    42. A H C Eaves & B G Kingsman, 2004. "Forecasting for the ordering and stock-holding of spare parts," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 55(4), pages 431-437, April.
    43. Acar, Yavuz & Gardner, Everette S., 2012. "Forecasting method selection in a global supply chain," International Journal of Forecasting, Elsevier, vol. 28(4), pages 842-848.
    44. Babai, M. Zied & Ali, Mohammad M. & Nikolopoulos, Konstantinos, 2012. "Impact of temporal aggregation on stock control performance of intermittent demand estimators: Empirical analysis," Omega, Elsevier, vol. 40(6), pages 713-721.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Li Li & Yanfei Kang & Fotios Petropoulos & Feng Li, 2022. "Feature-based intermittent demand forecast combinations: bias, accuracy and inventory implications," Papers 2204.08283, arXiv.org, revised Aug 2022.
    2. Fotios Petropoulos & Evangelos Spiliotis, 2021. "The Wisdom of the Data: Getting the Most Out of Univariate Time Series Forecasting," Forecasting, MDPI, vol. 3(3), pages 1-20, June.
    3. Udenio, Maximiliano & Vatamidou, Eleni & Fransoo, Jan C., 2023. "Exponential smoothing forecasts: Taming the Bullwhip Effect when demand is seasonal," Other publications TiSEM 8fca6329-83b9-4a49-a2aa-e, Tilburg University, School of Economics and Management.
    4. Melchior, Cristiane & Zanini, Roselaine Ruviaro & Guerra, Renata Rojas & Rockenbach, Dinei A., 2021. "Forecasting Brazilian mortality rates due to occupational accidents using autoregressive moving average approaches," International Journal of Forecasting, Elsevier, vol. 37(2), pages 825-837.
    5. Kang, Yanfei & Cao, Wei & Petropoulos, Fotios & Li, Feng, 2022. "Forecast with forecasts: Diversity matters," European Journal of Operational Research, Elsevier, vol. 301(1), pages 180-190.
    6. Liu, Zhenyuan & Han, Shuihua & Li, Chao & Gupta, Shivam & Sivarajah, Uthayasankar, 2022. "Leveraging customer engagement to improve the operational efficiency of social commerce start-ups," Journal of Business Research, Elsevier, vol. 140(C), pages 572-582.
    7. Meira, Erick & Cyrino Oliveira, Fernando Luiz & de Menezes, Lilian M., 2022. "Forecasting natural gas consumption using Bagging and modified regularization techniques," Energy Economics, Elsevier, vol. 106(C).
    8. Rostami-Tabar, Bahman & Disney, Stephen M., 2023. "On the order-up-to policy with intermittent integer demand and logically consistent forecasts," International Journal of Production Economics, Elsevier, vol. 257(C).
    9. Fotios Petropoulos & Enno Siemsen, 2023. "Forecast Selection and Representativeness," Management Science, INFORMS, vol. 69(5), pages 2672-2690, May.
    10. Petropoulos, Fotios & Svetunkov, Ivan, 2020. "A simple combination of univariate models," International Journal of Forecasting, Elsevier, vol. 36(1), pages 110-115.
    11. Kück, Mirko & Freitag, Michael, 2021. "Forecasting of customer demands for production planning by local k-nearest neighbor models," International Journal of Production Economics, Elsevier, vol. 231(C).
    12. Makridakis, Spyros & Spiliotis, Evangelos & Assimakopoulos, Vassilios, 2020. "The M4 Competition: 100,000 time series and 61 forecasting methods," International Journal of Forecasting, Elsevier, vol. 36(1), pages 54-74.
    13. Ducharme, Corey & Agard, Bruno & Trépanier, Martin, 2021. "Forecasting a customer's Next Time Under Safety Stock," International Journal of Production Economics, Elsevier, vol. 234(C).
    14. Spiliotis, Evangelos & Kouloumos, Andreas & Assimakopoulos, Vassilios & Makridakis, Spyros, 2020. "Are forecasting competitions data representative of the reality?," International Journal of Forecasting, Elsevier, vol. 36(1), pages 37-53.
    15. Spyros Makridakis & Chris Fry & Fotios Petropoulos & Evangelos Spiliotis, 2022. "The Future of Forecasting Competitions: Design Attributes and Principles," INFORMS Joural on Data Science, INFORMS, vol. 1(1), pages 96-113, April.
    16. George Athanasopoulos & Rob J Hyndman & Nikolaos Kourentzes & Anastasios Panagiotelis, 2023. "Forecast Reconciliation: A Review," Monash Econometrics and Business Statistics Working Papers 8/23, Monash University, Department of Econometrics and Business Statistics.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    2. Pinçe, Çerağ & Turrini, Laura & Meissner, Joern, 2021. "Intermittent demand forecasting for spare parts: A Critical review," Omega, Elsevier, vol. 105(C).
    3. Kourentzes, Nikolaos, 2014. "On intermittent demand model optimisation and selection," International Journal of Production Economics, Elsevier, vol. 156(C), pages 180-190.
    4. Hu, Qiwei & Boylan, John E. & Chen, Huijing & Labib, Ashraf, 2018. "OR in spare parts management: A review," European Journal of Operational Research, Elsevier, vol. 266(2), pages 395-414.
    5. Evangelos Spiliotis & Spyros Makridakis & Artemios-Anargyros Semenoglou & Vassilios Assimakopoulos, 2022. "Comparison of statistical and machine learning methods for daily SKU demand forecasting," Operational Research, Springer, vol. 22(3), pages 3037-3061, July.
    6. R Fildes & K Nikolopoulos & S F Crone & A A Syntetos, 2008. "Forecasting and operational research: a review," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 59(9), pages 1150-1172, September.
    7. Kang, Yanfei & Spiliotis, Evangelos & Petropoulos, Fotios & Athiniotis, Nikolaos & Li, Feng & Assimakopoulos, Vassilios, 2021. "Déjà vu: A data-centric forecasting approach through time series cross-similarity," Journal of Business Research, Elsevier, vol. 132(C), pages 719-731.
    8. Babai, Zied & Boylan, John E. & Kolassa, Stephan & Nikolopoulos, Konstantinos, 2016. "Supply chain forecasting: Theory, practice, their gap and the futureAuthor-Name: Syntetos, Aris A," European Journal of Operational Research, Elsevier, vol. 252(1), pages 1-26.
    9. Kourentzes, Nikolaos, 2013. "Intermittent demand forecasts with neural networks," International Journal of Production Economics, Elsevier, vol. 143(1), pages 198-206.
    10. Saoud, Patrick & Kourentzes, Nikolaos & Boylan, John E., 2022. "Approximations for the Lead Time Variance: a Forecasting and Inventory Evaluation," Omega, Elsevier, vol. 110(C).
    11. Tian, Xin & Wang, Haoqing & E, Erjiang, 2021. "Forecasting intermittent demand for inventory management by retailers: A new approach," Journal of Retailing and Consumer Services, Elsevier, vol. 62(C).
    12. Kourentzes, Nikolaos & Trapero, Juan R. & Barrow, Devon K., 2020. "Optimising forecasting models for inventory planning," International Journal of Production Economics, Elsevier, vol. 225(C).
    13. Makridakis, Spyros & Spiliotis, Evangelos & Assimakopoulos, Vassilios, 2022. "M5 accuracy competition: Results, findings, and conclusions," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1346-1364.
    14. Fotios Petropoulos & Evangelos Spiliotis, 2021. "The Wisdom of the Data: Getting the Most Out of Univariate Time Series Forecasting," Forecasting, MDPI, vol. 3(3), pages 1-20, June.
    15. Petropoulos, Fotios & Makridakis, Spyros & Assimakopoulos, Vassilios & Nikolopoulos, Konstantinos, 2014. "‘Horses for Courses’ in demand forecasting," European Journal of Operational Research, Elsevier, vol. 237(1), pages 152-163.
    16. Gardner, Everette Jr., 2006. "Exponential smoothing: The state of the art--Part II," International Journal of Forecasting, Elsevier, vol. 22(4), pages 637-666.
    17. Pennings, Clint L.P. & van Dalen, Jan & van der Laan, Erwin A., 2017. "Exploiting elapsed time for managing intermittent demand for spare parts," European Journal of Operational Research, Elsevier, vol. 258(3), pages 958-969.
    18. Kourentzes, Nikolaos & Athanasopoulos, George, 2021. "Elucidate structure in intermittent demand series," European Journal of Operational Research, Elsevier, vol. 288(1), pages 141-152.
    19. Babai, M.Z. & Ali, M.M. & Boylan, J.E. & Syntetos, A.A., 2013. "Forecasting and inventory performance in a two-stage supply chain with ARIMA(0,1,1) demand: Theory and empirical analysis," International Journal of Production Economics, Elsevier, vol. 143(2), pages 463-471.
    20. Van der Auweraer, Sarah & Boute, Robert, 2019. "Forecasting spare part demand using service maintenance information," International Journal of Production Economics, Elsevier, vol. 213(C), pages 138-149.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:intfor:v:35:y:2019:i:1:p:251-265. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/ijforecast .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.