IDEAS home Printed from https://ideas.repec.org/p/wpa/wuwpgt/0412002.html
   My bibliography  Save this paper

Correspondence On the Selection of Error Measures for Comparisons Among Forecasting Methods

Author

Listed:
  • JS Armstrong

    (The Wharton School - University of Pennsylvania)

  • Robert Fildes

    (The Management School - Lancaster University - UK)

Abstract

Clements and Hendry (1993) proposed the Generalized Forecast Error Second Moment (GFESM) as an improvement to the Mean Square Error in comparing forecasting performance across data series. They based their conclusion on the fact that rankings based on GFESM remain unaltered if the series are linearly transformed. In this paper, we argue that this evaluation ignores other important criteria. Also, their conclusions were illustrated by a simulation study whose relationship to real data was not obvious. Thirdly, prior empirical studies show that the mean square error is an inappropriate measure to serve as a basis for comparison. This undermines the claims made for the GFESM.

Suggested Citation

  • JS Armstrong & Robert Fildes, 2004. "Correspondence On the Selection of Error Measures for Comparisons Among Forecasting Methods," General Economics and Teaching 0412002, University Library of Munich, Germany.
  • Handle: RePEc:wpa:wuwpgt:0412002
    Note: Type of Document - pdf; pages: 4
    as

    Download full text from publisher

    File URL: https://econwpa.ub.uni-muenchen.de/econ-wp/get/papers/0412/0412002.pdf
    Download Restriction: no

    References listed on IDEAS

    as
    1. Clements, M.P. & Hendry, D., 1992. "On the Limitations of Comparing Mean Square Forecast Errors," Economics Series Working Papers 99138, University of Oxford, Department of Economics.
    2. Makridakis, Spyros & Hibon, Michele & Lusk, Ed & Belhadjali, Moncef, 1987. "Confidence intervals: An empirical investigation of the series in the M-competition," International Journal of Forecasting, Elsevier, vol. 3(3-4), pages 489-508.
    3. Robert Carbone & JS Armstrong, 2004. "Evaluation of Extrapolative Forecasting Methods: Results of a Survey of Academicians and Practitioners," General Economics and Teaching 0412008, University Library of Munich, Germany.
    4. Murphy, Allan H. & Winkler, Robert L., 1992. "Diagnostic verification of probability forecasts," International Journal of Forecasting, Elsevier, vol. 7(4), pages 435-455, March.
    5. Armstrong, J. Scott & Collopy, Fred, 1992. "Error measures for generalizing about forecasting methods: Empirical comparisons," International Journal of Forecasting, Elsevier, vol. 8(1), pages 69-80, June.
    6. Thompson, Patrick A., 1990. "An MSE statistic for comparing forecast accuracy across series," International Journal of Forecasting, Elsevier, vol. 6(2), pages 219-227, July.
    7. Fildes, Robert, 1992. "The evaluation of extrapolative forecasting methods," International Journal of Forecasting, Elsevier, vol. 8(1), pages 81-98, June.
    8. Zellner, Arnold, 1986. "A tale of forecasting 1001 series : The Bayesian knight strikes again," International Journal of Forecasting, Elsevier, vol. 2(4), pages 491-494.
    Full references (including those not matched with items on IDEAS)

    More about this item

    Keywords

    Accuracy Forecast evaluation Loss functions;

    JEL classification:

    • A - General Economics and Teaching

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wpa:wuwpgt:0412002. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (EconWPA). General contact details of provider: https://econwpa.ub.uni-muenchen.de .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.