IDEAS home Printed from https://ideas.repec.org/a/eee/intfor/v32y2016i3p788-803.html
   My bibliography  Save this article

Evaluating predictive count data distributions in retail sales forecasting

Author

Listed:
  • Kolassa, Stephan

Abstract

Massive increases in computing power and new database architectures allow data to be stored and processed at finer and finer granularities, yielding count data time series with lower and lower counts. These series can no longer be dealt with using the approximative methods that are appropriate for continuous probability distributions. In addition, it is not sufficient to calculate point forecasts alone: we need to forecast entire (discrete) predictive distributions, particularly for supply chain forecasting and inventory control, but also for other planning processes. However, tools that are suitable for evaluating the quality of discrete predictive distributions are not commonly used in sales forecasting. We explore classical point forecast accuracy measures, explain why measures such as MAD, MASE and wMAPE are inherently unsuitable for count data, and use the randomized Probability Integral Transform (PIT) and proper scoring rules to compare the performances of multiple causal and noncausal forecasting models on two datasets of daily retail sales.

Suggested Citation

  • Kolassa, Stephan, 2016. "Evaluating predictive count data distributions in retail sales forecasting," International Journal of Forecasting, Elsevier, vol. 32(3), pages 788-803.
  • Handle: RePEc:eee:intfor:v:32:y:2016:i:3:p:788-803
    DOI: 10.1016/j.ijforecast.2015.12.004
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0169207016000315
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ijforecast.2015.12.004?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Gneiting, Tilmann, 2011. "Making and Evaluating Point Forecasts," Journal of the American Statistical Association, American Statistical Association, vol. 106(494), pages 746-762.
    2. A A Syntetos & J E Boylan & J D Croston, 2005. "On the categorization of demand patterns," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 56(5), pages 495-503, May.
    3. Fotios Petropoulos & Nikolaos Kourentzes, 2015. "Forecast combinations for intermittent demand," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 66(6), pages 914-924, June.
    4. Corradi, Valentina & Swanson, Norman R., 2006. "Bootstrap conditional distribution tests in the presence of dynamic misspecification," Journal of Econometrics, Elsevier, vol. 133(2), pages 779-806, August.
    5. Willemain, Thomas R. & Smart, Charles N. & Schwarz, Henry, 2005. "Author's response to Koehler and Gardner," International Journal of Forecasting, Elsevier, vol. 21(3), pages 619-620.
    6. John Boylan & Aris Syntetos, 2006. "Accuracy and Accuracy Implication Metrics for Intermittent Demand," Foresight: The International Journal of Applied Forecasting, International Institute of Forecasters, issue 4, pages 39-42, June.
    7. Tim Januschowski & Stephan Kolassa & Martin Lorenz & Christian Schwarz, 2013. "Forecasting with In-Memory Technology," Foresight: The International Journal of Applied Forecasting, International Institute of Forecasters, issue 31, pages 14-20, Fall.
    8. Wallström, Peter & Segerstedt, Anders, 2010. "Evaluation of forecasting error measurements and techniques for intermittent demand," International Journal of Production Economics, Elsevier, vol. 128(2), pages 625-636, December.
    9. González-Rivera, Gloria & Sun, Yingying, 2015. "Generalized autocontours: Evaluation of multivariate density models," International Journal of Forecasting, Elsevier, vol. 31(3), pages 799-814.
    10. Snyder, Ralph D. & Ord, J. Keith & Beaumont, Adrian, 2012. "Forecasting the intermittent demand for slow-moving inventories: A modelling approach," International Journal of Forecasting, Elsevier, vol. 28(2), pages 485-496.
    11. Michael P. Clements & Nick Taylor, 2003. "Evaluating interval forecasts of high-frequency financial data," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 18(4), pages 445-456.
    12. Stephan Kolassa & Roland Martin, 2011. "Percentage Errors Can Ruin Your Day (and Rolling the Dice Shows How)," Foresight: The International Journal of Applied Forecasting, International Institute of Forecasters, issue 23, pages 21-27, Fall.
    13. A. S. C. Ehrenberg, 1959. "The Pattern of Consumer Purchases," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 8(1), pages 26-41, March.
    14. Diebold, Francis X & Gunther, Todd A & Tay, Anthony S, 1998. "Evaluating Density Forecasts with Applications to Financial Risk Management," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 863-883, November.
    15. Amisano, Gianni & Giacomini, Raffaella, 2007. "Comparing Density Forecasts via Weighted Likelihood Ratio Tests," Journal of Business & Economic Statistics, American Statistical Association, vol. 25, pages 177-190, April.
    16. Syntetos, Aris A. & Zied Babai, M. & Gardner, Everette S., 2015. "Forecasting intermittent inventory demands: simple parametric methods vs. bootstrapping," Journal of Business Research, Elsevier, vol. 68(8), pages 1746-1752.
    17. Hyndman, Rob J. & Koehler, Anne B., 2006. "Another look at measures of forecast accuracy," International Journal of Forecasting, Elsevier, vol. 22(4), pages 679-688.
    18. Christoffersen, Peter F, 1998. "Evaluating Interval Forecasts," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 841-862, November.
    19. Wallis, Kenneth F., 2003. "Chi-squared tests of interval and density forecasts, and the Bank of England's fan charts," International Journal of Forecasting, Elsevier, vol. 19(2), pages 165-175.
    20. Wei Wei & Leonhard Held, 2014. "Calibration tests for count data," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 23(4), pages 787-805, December.
    21. Willemain, Thomas R. & Smart, Charles N. & Schwarz, Henry F., 2004. "A new approach to forecasting intermittent demand for service parts inventories," International Journal of Forecasting, Elsevier, vol. 20(3), pages 375-387.
    22. Goodwin, Paul & Lawton, Richard, 1999. "On the asymmetry of the symmetric MAPE," International Journal of Forecasting, Elsevier, vol. 15(4), pages 405-408, October.
    23. Roman Liesenfeld & Ingmar Nolte & Winfried Pohlmeier, 2008. "Modelling financial transaction price movements: a dynamic integer count data model," Studies in Empirical Economics, in: Luc Bauwens & Winfried Pohlmeier & David Veredas (ed.), High Frequency Financial Econometrics, pages 167-197, Springer.
    24. Anthony Tay & Kenneth F. Wallis, 2000. "Density Forecasting: A Survey," Econometric Society World Congress 2000 Contributed Papers 0370, Econometric Society.
    25. Syntetos, Aris A. & Boylan, John E., 2005. "The accuracy of intermittent demand estimates," International Journal of Forecasting, Elsevier, vol. 21(2), pages 303-314.
    26. Tilmann Gneiting & Fadoua Balabdaoui & Adrian E. Raftery, 2007. "Probabilistic forecasts, calibration and sharpness," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 69(2), pages 243-268, April.
    27. Tilmann Gneiting & Roopesh Ranjan, 2011. "Comparing Density Forecasts Using Threshold- and Quantile-Weighted Scoring Rules," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 29(3), pages 411-422, July.
    28. Claudia Czado & Tilmann Gneiting & Leonhard Held, 2009. "Predictive Model Assessment for Count Data," Biometrics, The International Biometric Society, vol. 65(4), pages 1254-1261, December.
    29. Kourentzes, Nikolaos, 2014. "On intermittent demand model optimisation and selection," International Journal of Production Economics, Elsevier, vol. 156(C), pages 180-190.
    30. Franses, Philip Hans, 2016. "A note on the Mean Absolute Scaled Error," International Journal of Forecasting, Elsevier, vol. 32(1), pages 20-22.
    31. Axel Munk & Jean-Pierre Stockis & Janis Valeinis & Götz Giese, 2011. "Neyman smooth goodness-of-fit tests for the marginal distribution of dependent data," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 63(5), pages 939-959, October.
    32. Kampstra, Peter, 2008. "Beanplot: A Boxplot Alternative for Visual Comparison of Distributions," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 28(c01).
    33. Steve Morlidge, 2015. "Measuring the Quality of Intermittent-Demand Forecasts: ItÕs Worse than WeÕve Thought!," Foresight: The International Journal of Applied Forecasting, International Institute of Forecasters, issue 37, pages 37-42, Spring.
    34. Rob J. Hyndman, 2006. "Another Look at Forecast Accuracy Metrics for Intermittent Demand," Foresight: The International Journal of Applied Forecasting, International Institute of Forecasters, issue 4, pages 43-46, June.
    35. Gneiting, Tilmann & Ranjan, Roopesh, 2011. "Comparing Density Forecasts Using Threshold- and Quantile-Weighted Scoring Rules," Journal of Business & Economic Statistics, American Statistical Association, vol. 29(3), pages 411-422.
    36. J E Boylan & A A Syntetos & G C Karakostas, 2008. "Classification for forecasting and stock control: a case study," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 59(4), pages 473-481, April.
    37. James Mitchell & Kenneth F. Wallis, 2011. "Evaluating density forecasts: forecast combinations, model mixtures, calibration and sharpness," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 26(6), pages 1023-1040, September.
    38. Hanley J. A. & Joseph L. & Platt R. W. & Chung M. K. & Belisle P., 2001. "Visualizing the Median as the Minimum-Deviation Location," The American Statistician, American Statistical Association, vol. 55, pages 150-152, May.
    39. R H Teunter & L Duncan, 2009. "Forecasting intermittent demand: a comparative study," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 60(3), pages 321-329, March.
    40. Stephan Kolassa, 2016. "Sometimes It's Better to Be Simple than Correct," Foresight: The International Journal of Applied Forecasting, International Institute of Forecasters, issue 40, pages 20-26, Winter.
    41. Tom Willemain, 2006. "Forecast Accuracy Metrics for Intermittent Demands: Look at the Entire Distribution of Demands," Foresight: The International Journal of Applied Forecasting, International Institute of Forecasters, issue 4, pages 36-38, June.
    42. Hajo Holzmann & Matthias Eulert, 2014. "The role of the information set for forecasting - with applications to risk management," Papers 1404.7653, arXiv.org.
    43. Brockwell, A.E., 2007. "Universal residuals: A multivariate transformation," Statistics & Probability Letters, Elsevier, vol. 77(14), pages 1473-1478, August.
    44. Stephan Kolassa & Wolfgang Schütz, 2007. "Advantages of the MAD/Mean Ratio over the MAPE," Foresight: The International Journal of Applied Forecasting, International Institute of Forecasters, issue 6, pages 40-43, Spring.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    2. Mariusz Doszyn, 2020. "Accuracy of Intermittent Demand Forecasting Systems in the Enterprise," European Research Studies Journal, European Research Studies Journal, vol. 0(4), pages 912-930.
    3. Ducharme, Corey & Agard, Bruno & Trépanier, Martin, 2021. "Forecasting a customer's Next Time Under Safety Stock," International Journal of Production Economics, Elsevier, vol. 234(C).
    4. Pinçe, Çerağ & Turrini, Laura & Meissner, Joern, 2021. "Intermittent demand forecasting for spare parts: A Critical review," Omega, Elsevier, vol. 105(C).
    5. Fildes, Robert & Ma, Shaohui & Kolassa, Stephan, 2022. "Retail forecasting: Research and practice," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1283-1318.
    6. Marc-Oliver Pohle, 2020. "The Murphy Decomposition and the Calibration-Resolution Principle: A New Perspective on Forecast Evaluation," Papers 2005.01835, arXiv.org.
    7. Evangelos Spiliotis & Spyros Makridakis & Artemios-Anargyros Semenoglou & Vassilios Assimakopoulos, 2022. "Comparison of statistical and machine learning methods for daily SKU demand forecasting," Operational Research, Springer, vol. 22(3), pages 3037-3061, July.
    8. Sarlo, Rodrigo & Fernandes, Cristiano & Borenstein, Denis, 2023. "Lumpy and intermittent retail demand forecasts with score-driven models," European Journal of Operational Research, Elsevier, vol. 307(3), pages 1146-1160.
    9. Kourentzes, Nikolaos & Athanasopoulos, George, 2021. "Elucidate structure in intermittent demand series," European Journal of Operational Research, Elsevier, vol. 288(1), pages 141-152.
    10. Tsyplakov, Alexander, 2013. "Evaluation of Probabilistic Forecasts: Proper Scoring Rules and Moments," MPRA Paper 45186, University Library of Munich, Germany.
    11. Gordy, Michael B. & McNeil, Alexander J., 2020. "Spectral backtests of forecast distributions with application to risk management," Journal of Banking & Finance, Elsevier, vol. 116(C).
    12. Altay, Nezih & Rudisill, Frank & Litteral, Lewis A., 2008. "Adapting Wright's modification of Holt's method to forecasting intermittent demand," International Journal of Production Economics, Elsevier, vol. 111(2), pages 389-408, February.
    13. Weron, Rafał, 2014. "Electricity price forecasting: A review of the state-of-the-art with a look into the future," International Journal of Forecasting, Elsevier, vol. 30(4), pages 1030-1081.
    14. Jože Martin Rožanec & Blaž Fortuna & Dunja Mladenić, 2022. "Reframing Demand Forecasting: A Two-Fold Approach for Lumpy and Intermittent Demand," Sustainability, MDPI, vol. 14(15), pages 1-21, July.
    15. Mariusz Doszyn, 2020. "Biasedness of Forecasts Errors for Intermittent Demand Data," European Research Studies Journal, European Research Studies Journal, vol. 0(Special 1), pages 1113-1127.
    16. Tsyplakov, Alexander, 2014. "Theoretical guidelines for a partially informed forecast examiner," MPRA Paper 55017, University Library of Munich, Germany.
    17. Hu, Qiwei & Boylan, John E. & Chen, Huijing & Labib, Ashraf, 2018. "OR in spare parts management: A review," European Journal of Operational Research, Elsevier, vol. 266(2), pages 395-414.
    18. Philippe St-Aubin & Bruno Agard, 2022. "Precision and Reliability of Forecasts Performance Metrics," Forecasting, MDPI, vol. 4(4), pages 1-22, October.
    19. Costantino, Francesco & Di Gravio, Giulio & Patriarca, Riccardo & Petrella, Lea, 2018. "Spare parts management for irregular demand items," Omega, Elsevier, vol. 81(C), pages 57-66.
    20. Petropoulos, Fotios & Wang, Xun & Disney, Stephen M., 2019. "The inventory performance of forecasting methods: Evidence from the M3 competition data," International Journal of Forecasting, Elsevier, vol. 35(1), pages 251-265.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:intfor:v:32:y:2016:i:3:p:788-803. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/ijforecast .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.