IDEAS home Printed from https://ideas.repec.org/a/eee/intfor/v29y2013i3p510-522.html
   My bibliography  Save this article

Measuring forecasting accuracy: The case of judgmental adjustments to SKU-level demand forecasts

Author

Listed:
  • Davydenko, Andrey
  • Fildes, Robert

Abstract

Forecast adjustment commonly occurs when organizational forecasters adjust a statistical forecast of demand to take into account factors which are excluded from the statistical calculation. This paper addresses the question of how to measure the accuracy of such adjustments. We show that many existing error measures are generally not suited to the task, due to specific features of the demand data. Alongside the well-known weaknesses of existing measures, a number of additional effects are demonstrated that complicate the interpretation of measurement results and can even lead to false conclusions being drawn. In order to ensure an interpretable and unambiguous evaluation, we recommend the use of a metric based on aggregating performance ratios across time series using the weighted geometric mean. We illustrate that this measure has the advantage of treating over- and under-forecasting even-handedly, has a more symmetric distribution, and is robust.

Suggested Citation

  • Davydenko, Andrey & Fildes, Robert, 2013. "Measuring forecasting accuracy: The case of judgmental adjustments to SKU-level demand forecasts," International Journal of Forecasting, Elsevier, vol. 29(3), pages 510-522.
  • Handle: RePEc:eee:intfor:v:29:y:2013:i:3:p:510-522
    DOI: 10.1016/j.ijforecast.2012.09.002
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0169207012001161
    Download Restriction: Full text for ScienceDirect subscribers only

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Hyndman, Rob J. & Khandakar, Yeasmin, 2008. "Automatic Time Series Forecasting: The forecast Package for R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 27(i03).
    2. Donna F. Davis & John T. Mentzer & Teresa M. Mccarthy & Susan L. Golicic, 2006. "The evolution of sales forecasting management: a 20-year longitudinal study of forecasting practices," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 25(5), pages 303-324.
    3. Syntetos, Aris A. & Boylan, John E., 2005. "The accuracy of intermittent demand estimates," International Journal of Forecasting, Elsevier, vol. 21(2), pages 303-314.
    4. Philip Hans Franses & Rianne Legerstee, 2010. "Do experts' adjustments on model-based SKU-level forecasts improve forecast quality?," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 29(3), pages 331-340.
    5. Robalo Marques, Carlos & Duarte Neves, Pedro & Morais Sarmento, Luis, 2003. "Evaluating core inflation indicators," Economic Modelling, Elsevier, vol. 20(4), pages 765-775, July.
    6. Rob J. Hyndman, 2006. "Another Look at Forecast Accuracy Metrics for Intermittent Demand," Foresight: The International Journal of Applied Forecasting, International Institute of Forecasters, issue 4, pages 43-46, June.
    7. Hyndman, Rob J. & Koehler, Anne B., 2006. "Another look at measures of forecast accuracy," International Journal of Forecasting, Elsevier, vol. 22(4), pages 679-688.
    8. Armstrong, J. Scott & Collopy, Fred, 1992. "Error measures for generalizing about forecasting methods: Empirical comparisons," International Journal of Forecasting, Elsevier, vol. 8(1), pages 69-80, June.
    9. Goodwin, Paul & Lawton, Richard, 1999. "On the asymmetry of the symmetric MAPE," International Journal of Forecasting, Elsevier, vol. 15(4), pages 405-408, October.
    10. Fildes, Robert & Goodwin, Paul & Lawrence, Michael & Nikolopoulos, Konstantinos, 2009. "Effective forecasting and judgmental adjustments: an empirical evaluation and strategies for improvement in supply-chain planning," International Journal of Forecasting, Elsevier, vol. 25(1), pages 3-23.
    11. Makridakis, Spyros, 1993. "Accuracy measures: theoretical and practical concerns," International Journal of Forecasting, Elsevier, vol. 9(4), pages 527-529, December.
    12. Fildes, Robert, 1992. "The evaluation of extrapolative forecasting methods," International Journal of Forecasting, Elsevier, vol. 8(1), pages 81-98, June.
    13. Zellner, Arnold, 1986. "A tale of forecasting 1001 series : The Bayesian knight strikes again," International Journal of Forecasting, Elsevier, vol. 2(4), pages 491-494.
    14. Stephan Kolassa & Wolfgang Schütz, 2007. "Advantages of the MAD/Mean Ratio over the MAPE," Foresight: The International Journal of Applied Forecasting, International Institute of Forecasters, issue 6, pages 40-43, Spring.
    15. Jim Hoover, 2006. "Measuring Forecast Accuracy: Omissions in Today's Forecasting Engines and Demand-Planning Software," Foresight: The International Journal of Applied Forecasting, International Institute of Forecasters, issue 4, pages 32-35, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. repec:eee:ejores:v:264:y:2018:i:3:p:967-977 is not listed on IDEAS
    2. Sbrana, Giacomo & Silvestrini, Andrea, 2013. "Forecasting aggregate demand: Analytical comparison of top-down and bottom-up approaches in a multivariate exponential smoothing framework," International Journal of Production Economics, Elsevier, vol. 146(1), pages 185-198.
    3. Meeran, Sheik & Jahanbin, Semco & Goodwin, Paul & Quariguasi Frota Neto, Joao, 2017. "When do changes in consumer preferences make forecasts from choice-based conjoint models unreliable?," European Journal of Operational Research, Elsevier, vol. 258(2), pages 512-524.
    4. Wan, Xiang & Sanders, Nadia R., 2017. "The negative impact of product variety: Forecast bias, inventory levels, and the role of vertical integration," International Journal of Production Economics, Elsevier, vol. 186(C), pages 123-131.
    5. Athanasopoulos, George & Hyndman, Rob J. & Kourentzes, Nikolaos & Petropoulos, Fotios, 2017. "Forecasting with temporal hierarchies," European Journal of Operational Research, Elsevier, vol. 262(1), pages 60-74.
    6. Victor Richmond R. Jose, 2017. "Percentage and Relative Error Measures in Forecast Evaluation," Operations Research, INFORMS, vol. 65(1), pages 200-211, February.
    7. Huang, Tao & Fildes, Robert & Soopramanien, Didier, 2014. "The value of competitive information in forecasting FMCG retail product sales and the variable selection problem," European Journal of Operational Research, Elsevier, vol. 237(2), pages 738-748.
    8. repec:eee:intfor:v:33:y:2017:i:3:p:652-661 is not listed on IDEAS
    9. Ma, Shaohui & Fildes, Robert, 2017. "A retail store SKU promotions optimization model for category multi-period profit maximization," European Journal of Operational Research, Elsevier, vol. 260(2), pages 680-692.
    10. Bentour, El Mostafa, 2015. "A ranking of VAR and structural models in forecasting," MPRA Paper 61502, University Library of Munich, Germany.
    11. Hartzel, Kathleen S. & Wood, Charles A., 2017. "Factors that affect the improvement of demand forecast accuracy through point-of-sale reporting," European Journal of Operational Research, Elsevier, vol. 260(1), pages 171-182.
    12. Fildes, Robert & Petropoulos, Fotios, 2015. "Is there a Golden Rule?," Journal of Business Research, Elsevier, vol. 68(8), pages 1742-1745.
    13. Petropoulos, Fotios & Fildes, Robert & Goodwin, Paul, 2016. "Do ‘big losses’ in judgmental adjustments to statistical forecasts affect experts’ behaviour?," European Journal of Operational Research, Elsevier, vol. 249(3), pages 842-852.
    14. Fildes, Robert & Petropoulos, Fotios, 2015. "Simple versus complex selection rules for forecasting many time series," Journal of Business Research, Elsevier, vol. 68(8), pages 1692-1701.
    15. Fildes, Robert & Petropoulos, Fotios, 2013. "An evaluation of simple forecasting model selection rules," MPRA Paper 51772, University Library of Munich, Germany.
    16. Ma, Shaohui & Fildes, Robert & Huang, Tao, 2016. "Demand forecasting with high dimensional data: The case of SKU retail sales forecasting with intra- and inter-category promotional information," European Journal of Operational Research, Elsevier, vol. 249(1), pages 245-257.
    17. Emrouznejad, Ali & Rostami-Tabar, Bahman & Petridis, Konstantinos, 2016. "A novel ranking procedure for forecasting approaches using Data Envelopment Analysis," Technological Forecasting and Social Change, Elsevier, vol. 111(C), pages 235-243.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:intfor:v:29:y:2013:i:3:p:510-522. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Dana Niculescu). General contact details of provider: http://www.elsevier.com/locate/ijforecast .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.