Advanced Search
MyIDEAS: Login

Measuring forecasting accuracy: The case of judgmental adjustments to SKU-level demand forecasts

Contents:

Author Info

  • Davydenko, Andrey
  • Fildes, Robert
Registered author(s):

    Abstract

    Forecast adjustment commonly occurs when organizational forecasters adjust a statistical forecast of demand to take into account factors which are excluded from the statistical calculation. This paper addresses the question of how to measure the accuracy of such adjustments. We show that many existing error measures are generally not suited to the task, due to specific features of the demand data. Alongside the well-known weaknesses of existing measures, a number of additional effects are demonstrated that complicate the interpretation of measurement results and can even lead to false conclusions being drawn. In order to ensure an interpretable and unambiguous evaluation, we recommend the use of a metric based on aggregating performance ratios across time series using the weighted geometric mean. We illustrate that this measure has the advantage of treating over- and under-forecasting even-handedly, has a more symmetric distribution, and is robust.

    Download Info

    If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
    File URL: http://www.sciencedirect.com/science/article/pii/S0169207012001161
    Download Restriction: Full text for ScienceDirect subscribers only

    As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.

    Bibliographic Info

    Article provided by Elsevier in its journal International Journal of Forecasting.

    Volume (Year): 29 (2013)
    Issue (Month): 3 ()
    Pages: 510-522

    as in new window
    Handle: RePEc:eee:intfor:v:29:y:2013:i:3:p:510-522

    Contact details of provider:
    Web page: http://www.elsevier.com/locate/ijforecast

    Related research

    Keywords: Judgmental adjustments; Forecasting support systems; Forecast accuracy; Forecast evaluation; Forecast error measures;

    References

    References listed on IDEAS
    Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
    as in new window
    1. Jim Hoover, 2006. "Measuring Forecast Accuracy: Omissions in Today's Forecasting Engines and Demand-Planning Software," Foresight: The International Journal of Applied Forecasting, International Institute of Forecasters, issue 4, pages 32-35, June.
    2. Fildes, Robert, 1992. "The evaluation of extrapolative forecasting methods," International Journal of Forecasting, Elsevier, vol. 8(1), pages 81-98, June.
    3. Makridakis, Spyros, 1993. "Accuracy measures: theoretical and practical concerns," International Journal of Forecasting, Elsevier, vol. 9(4), pages 527-529, December.
    4. Pedro Duarte Neves & Luís Morais Sarmento & Carlos Robalo Marques, 1999. "Evaluating core inflation indicators," Economic Bulletin and Financial Stability Report Articles, Banco de Portugal, Economics and Research Department.
    5. Goodwin, Paul & Lawton, Richard, 1999. "On the asymmetry of the symmetric MAPE," International Journal of Forecasting, Elsevier, vol. 15(4), pages 405-408, October.
    6. Rob J. Hyndman & Anne B. Koehler, 2005. "Another Look at Measures of Forecast Accuracy," Monash Econometrics and Business Statistics Working Papers 13/05, Monash University, Department of Econometrics and Business Statistics.
    7. Donna F. Davis & John T. Mentzer & Teresa M. Mccarthy & Susan L. Golicic, 2006. "The evolution of sales forecasting management: a 20-year longitudinal study of forecasting practices," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 25(5), pages 303-324.
    8. Rob J. Hyndman, 2006. "Another Look at Forecast Accuracy Metrics for Intermittent Demand," Foresight: The International Journal of Applied Forecasting, International Institute of Forecasters, issue 4, pages 43-46, June.
    9. Fildes, Robert & Goodwin, Paul & Lawrence, Michael & Nikolopoulos, Konstantinos, 2009. "Effective forecasting and judgmental adjustments: an empirical evaluation and strategies for improvement in supply-chain planning," International Journal of Forecasting, Elsevier, vol. 25(1), pages 3-23.
    10. Stephan Kolassa & Wolfgang Schütz, 2007. "Advantages of the MAD/Mean Ratio over the MAPE," Foresight: The International Journal of Applied Forecasting, International Institute of Forecasters, issue 6, pages 40-43, Spring.
    11. Rob J. Hyndman & Yeasmin Khandakar, . "Automatic Time Series Forecasting: The forecast Package for R," Journal of Statistical Software, American Statistical Association, vol. 27(i03).
    12. Armstrong, J. Scott & Collopy, Fred, 1992. "Error measures for generalizing about forecasting methods: Empirical comparisons," International Journal of Forecasting, Elsevier, vol. 8(1), pages 69-80, June.
    13. Syntetos, Aris A. & Boylan, John E., 2005. "The accuracy of intermittent demand estimates," International Journal of Forecasting, Elsevier, vol. 21(2), pages 303-314.
    14. Zellner, Arnold, 1986. "A tale of forecasting 1001 series : The Bayesian knight strikes again," International Journal of Forecasting, Elsevier, vol. 2(4), pages 491-494.
    15. Philip Hans Franses & Rianne Legerstee, 2010. "Do experts' adjustments on model-based SKU-level forecasts improve forecast quality?," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 29(3), pages 331-340.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as in new window

    Cited by:
    1. Sbrana, Giacomo & Silvestrini, Andrea, 2013. "Forecasting aggregate demand: Analytical comparison of top-down and bottom-up approaches in a multivariate exponential smoothing framework," International Journal of Production Economics, Elsevier, vol. 146(1), pages 185-198.
    2. Fildes, Robert & Petropoulos, Fotios, 2013. "An evaluation of simple forecasting model selection rules," MPRA Paper 51772, University Library of Munich, Germany.

    Lists

    This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

    Statistics

    Access and download statistics

    Corrections

    When requesting a correction, please mention this item's handle: RePEc:eee:intfor:v:29:y:2013:i:3:p:510-522. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Zhang, Lei).

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If references are entirely missing, you can add them using this form.

    If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.