IDEAS home Printed from https://ideas.repec.org/p/msh/ebswps/2005-13.html
   My bibliography  Save this paper

Another Look at Measures of Forecast Accuracy

Author

Listed:
  • Rob J. Hyndman

    ()

  • Anne B. Koehler

Abstract

We discuss and compare measures of accuracy of univariate time series forecasts. The methods used in the M-competition and the M3-competition, and many of the measures recommended by previous authors on this topic, are found to be inadequate, and many of them are degenerate in commonly occurring situations. Instead, we propose that the mean absolute scaled error become the standard measure for comparing forecast accuracy across multiple time series.

Suggested Citation

  • Rob J. Hyndman & Anne B. Koehler, 2005. "Another Look at Measures of Forecast Accuracy," Monash Econometrics and Business Statistics Working Papers 13/05, Monash University, Department of Econometrics and Business Statistics.
  • Handle: RePEc:msh:ebswps:2005-13
    as

    Download full text from publisher

    File URL: http://www.buseco.monash.edu.au/ebs/pubs/wpapers/2005/wp13-05.pdf
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. Hyndman, Rob J. & Koehler, Anne B. & Snyder, Ralph D. & Grose, Simone, 2002. "A state space framework for automatic forecasting using exponential smoothing methods," International Journal of Forecasting, Elsevier, vol. 18(3), pages 439-454.
    2. Assimakopoulos, V. & Nikolopoulos, K., 2000. "The theta model: a decomposition approach to forecasting," International Journal of Forecasting, Elsevier, vol. 16(4), pages 521-530.
    3. Everette S. Gardner, 1990. "Evaluating Forecast Performance in an Inventory Control System," Management Science, INFORMS, vol. 36(4), pages 490-499, April.
    4. Billah, Baki & King, Maxwell L. & Snyder, Ralph D. & Koehler, Anne B., 2006. "Exponential smoothing model selection for forecasting," International Journal of Forecasting, Elsevier, vol. 22(2), pages 239-247.
    5. Goodwin, Paul & Lawton, Richard, 1999. "On the asymmetry of the symmetric MAPE," International Journal of Forecasting, Elsevier, vol. 15(4), pages 405-408, October.
    6. Chatfield, Chris, 1988. "Apples, oranges and mean square error," International Journal of Forecasting, Elsevier, vol. 4(4), pages 515-518.
    7. Makridakis, Spyros & Hibon, Michele, 2000. "The M3-Competition: results, conclusions and implications," International Journal of Forecasting, Elsevier, vol. 16(4), pages 451-476.
    8. Clements, M.P. & Hendry, D., 1992. "On the Limitations of Comparing Mean Square Forecast Errors," Economics Series Working Papers 99138, University of Oxford, Department of Economics.
    9. Hyndman, Rob J. & Billah, Baki, 2003. "Unmasking the Theta method," International Journal of Forecasting, Elsevier, vol. 19(2), pages 287-290.
    10. Lawrence, Michael & O'Connor, Marcus, 2005. "Judgmental forecasting in the presence of loss functions," International Journal of Forecasting, Elsevier, vol. 21(1), pages 3-14.
    11. Chatfield, Chris, 1992. "A commentary on error measures," International Journal of Forecasting, Elsevier, vol. 8(1), pages 100-102, June.
    12. Armstrong, J. Scott & Collopy, Fred, 1992. "Error measures for generalizing about forecasting methods: Empirical comparisons," International Journal of Forecasting, Elsevier, vol. 8(1), pages 69-80, June.
    13. Thompson, Patrick A., 1990. "An MSE statistic for comparing forecast accuracy across series," International Journal of Forecasting, Elsevier, vol. 6(2), pages 219-227, July.
    14. Koning, Alex J. & Franses, Philip Hans & Hibon, Michele & Stekler, H.O., 2005. "The M3 competition: Statistical tests of the results," International Journal of Forecasting, Elsevier, vol. 21(3), pages 397-409.
    15. Granger, C.W.J. & Pesaran, M. H., 1999. "Economic and Statistical Measures of Forecast Accuracy," Cambridge Working Papers in Economics 9910, Faculty of Economics, University of Cambridge.
    16. Makridakis, Spyros, 1993. "Accuracy measures: theoretical and practical concerns," International Journal of Forecasting, Elsevier, vol. 9(4), pages 527-529, December.
    17. Fildes, Robert, 1992. "The evaluation of extrapolative forecasting methods," International Journal of Forecasting, Elsevier, vol. 8(1), pages 81-98, June.
    Full references (including those not matched with items on IDEAS)

    More about this item

    Keywords

    Forecast accuracy; Forecast evaluation; Forecast error measures; M-competition; Mean absolute scaled error.;

    JEL classification:

    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes

    NEP fields

    This paper has been announced in the following NEP Reports:

    Lists

    This item is featured on the following reading lists or Wikipedia pages:
    1. Technology Assessment

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:msh:ebswps:2005-13. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Dr Xibin Zhang). General contact details of provider: http://edirc.repec.org/data/dxmonau.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.