Advanced Search
MyIDEAS: Login

Another Look at Measures of Forecast Accuracy

Contents:

Author Info

  • Rob J. Hyndman

    ()

  • Anne B. Koehler

Abstract

We discuss and compare measures of accuracy of univariate time series forecasts. The methods used in the M-competition and the M3-competition, and many of the measures recommended by previous authors on this topic, are found to be inadequate, and many of them are degenerate in commonly occurring situations. Instead, we propose that the mean absolute scaled error become the standard measure for comparing forecast accuracy across multiple time series.

Download Info

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
File URL: http://www.buseco.monash.edu.au/ebs/pubs/wpapers/2005/wp13-05.pdf
Download Restriction: no

Bibliographic Info

Paper provided by Monash University, Department of Econometrics and Business Statistics in its series Monash Econometrics and Business Statistics Working Papers with number 13/05.

as in new window
Length: 18 pages
Date of creation: May 2005
Date of revision:
Handle: RePEc:msh:ebswps:2005-13

Contact details of provider:
Postal: PO Box 11E, Monash University, Victoria 3800, Australia
Phone: +61-3-9905-2489
Fax: +61-3-9905-5474
Email:
Web page: http://www.buseco.monash.edu.au/depts/ebs/
More information through EDIRC

Order Information:
Email:
Web: http://www.buseco.monash.edu.au/depts/ebs/pubs/wpapers/

Related research

Keywords: Forecast accuracy; Forecast evaluation; Forecast error measures; M-competition; Mean absolute scaled error.;

Other versions of this item:

Find related papers by JEL classification:

This paper has been announced in the following NEP Reports:

References

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
as in new window
  1. Armstrong, J. Scott & Collopy, Fred, 1992. "Error measures for generalizing about forecasting methods: Empirical comparisons," International Journal of Forecasting, Elsevier, vol. 8(1), pages 69-80, June.
  2. Baki Billah & Maxwell L King & Ralph D Snyder & Anne B Koehler, 2005. "Exponential Smoothing Model Selection for Forecasting," Monash Econometrics and Business Statistics Working Papers 6/05, Monash University, Department of Econometrics and Business Statistics.
  3. Chatfield, Chris, 1992. "A commentary on error measures," International Journal of Forecasting, Elsevier, vol. 8(1), pages 100-102, June.
  4. Hyndman, Rob J. & Billah, Baki, 2003. "Unmasking the Theta method," International Journal of Forecasting, Elsevier, vol. 19(2), pages 287-290.
  5. Chatfield, Chris, 1988. "Apples, oranges and mean square error," International Journal of Forecasting, Elsevier, vol. 4(4), pages 515-518.
  6. Makridakis, Spyros, 1993. "Accuracy measures: theoretical and practical concerns," International Journal of Forecasting, Elsevier, vol. 9(4), pages 527-529, December.
  7. Hyndman, R.J. & Koehler, A.B. & Snyder, R.D. & Grose, S., 2000. "A State Space Framework for Automatic Forecasting Using Exponential Smoothing Methods," Monash Econometrics and Business Statistics Working Papers 9/00, Monash University, Department of Econometrics and Business Statistics.
  8. Thompson, Patrick A., 1990. "An MSE statistic for comparing forecast accuracy across series," International Journal of Forecasting, Elsevier, vol. 6(2), pages 219-227, July.
  9. Assimakopoulos, V. & Nikolopoulos, K., 2000. "The theta model: a decomposition approach to forecasting," International Journal of Forecasting, Elsevier, vol. 16(4), pages 521-530.
  10. Everette S. Gardner, 1990. "Evaluating Forecast Performance in an Inventory Control System," Management Science, INFORMS, vol. 36(4), pages 490-499, April.
  11. Goodwin, Paul & Lawton, Richard, 1999. "On the asymmetry of the symmetric MAPE," International Journal of Forecasting, Elsevier, vol. 15(4), pages 405-408, October.
  12. Makridakis, Spyros & Hibon, Michele, 2000. "The M3-Competition: results, conclusions and implications," International Journal of Forecasting, Elsevier, vol. 16(4), pages 451-476.
  13. Granger, C.W.J. & Pesaran, M. H., 1999. "Economic and Statistical Measures of Forecast Accuracy," Cambridge Working Papers in Economics 9910, Faculty of Economics, University of Cambridge.
  14. Koning, Alex J. & Franses, Philip Hans & Hibon, Michele & Stekler, H.O., 2005. "The M3 competition: Statistical tests of the results," International Journal of Forecasting, Elsevier, vol. 21(3), pages 397-409.
  15. Lawrence, Michael & O'Connor, Marcus, 2005. "Judgmental forecasting in the presence of loss functions," International Journal of Forecasting, Elsevier, vol. 21(1), pages 3-14.
  16. Clements, M.P. & Hendry, D., 1992. "On the Limitations of Comparing Mean Square Forecast Errors," Economics Series Working Papers 99138, University of Oxford, Department of Economics.
  17. Fildes, Robert, 1992. "The evaluation of extrapolative forecasting methods," International Journal of Forecasting, Elsevier, vol. 8(1), pages 81-98, June.
Full references (including those not matched with items on IDEAS)

Citations

Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
as in new window

Cited by:
This item has more than 25 citations. To prevent cluttering this page, these citations are listed on a separate page.

Lists

This item is featured on the following reading lists or Wikipedia pages:
  1. Technology Assessment

Statistics

Access and download statistics

Corrections

When requesting a correction, please mention this item's handle: RePEc:msh:ebswps:2005-13. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Simone Grose).

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.