A statistical test for forecast evaluation under a discrete loss function
AbstractWe propose a new approach to evaluating the usefulness of a set of forecasts, based on the use of a discrete loss function de ned on the space of data and forecasts. Exist- ing procedures for such an evaluation either do not allow for formal testing, or use tests statistics based just on the frequency distribution of (data , forecasts)-pairs. They can easily lead to misleading conclusions in some reasonable situations, because of the way they formalize the underlying null hypothesis that the set of forecasts is not useful. Even though the ambiguity of the underlying null hypothesis precludes us from per- forming a standard analysis of the size and power of the tests, we get results suggesting that the proposed DISC test performs better than its competitors.
Download InfoIf you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
Bibliographic InfoPaper provided by Universidad Complutense de Madrid, Facultad de Ciencias Económicas y Empresariales in its series Documentos del Instituto Complutense de Análisis Económico with number 2011-07.
Length: 17 pages
Date of creation: 2011
Date of revision:
Forecasting Evaluation; Loss Function.;
This paper has been announced in the following NEP Reports:
- NEP-ALL-2011-04-23 (All new papers)
- NEP-ECM-2011-04-23 (Econometrics)
- NEP-ETS-2011-04-23 (Econometric Time Series)
- NEP-FOR-2011-04-23 (Forecasting)
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Henriksson, Roy D & Merton, Robert C, 1981. "On Market Timing and Investment Performance. II. Statistical Procedures for Evaluating Forecasting Skills," The Journal of Business, University of Chicago Press, vol. 54(4), pages 513-33, October.
- Mills, Terence C. & Pepper, Gordon T., 1999. "Assessing the forecasters: an analysis of the forecasting records of the Treasury, the London Business School and the National Institute," International Journal of Forecasting, Elsevier, vol. 15(3), pages 247-257, July.
- Pesaran, M.H. & Timmermann, A., 1990.
"A Simple, Non-Parametric Test Of Predictive Performance,"
Cambridge Working Papers in Economics
9021, Faculty of Economics, University of Cambridge.
- Pesaran, M Hashem & Timmermann, Allan, 1992. "A Simple Nonparametric Test of Predictive Performance," Journal of Business & Economic Statistics, American Statistical Association, vol. 10(4), pages 561-65, October.
- Pesaran, M.H. & Timmermann, A., 1990. "A Simple Non-Parametric Test Of Predictive Performance," Papers 29, California Los Angeles - Applied Econometrics.
- Joutz, Fred & Stekler, H. O., 2000. "An evaluation of the predictions of the Federal Reserve," International Journal of Forecasting, Elsevier, vol. 16(1), pages 17-38.
- Merton, Robert C, 1981. "On Market Timing and Investment Performance. I. An Equilibrium Theory of Value for Market Forecasts," The Journal of Business, University of Chicago Press, vol. 54(3), pages 363-406, July.
- Ash, J. C. K. & Smyth, D. J. & Heravi, S. M., 1998. "Are OECD forecasts rational and useful?: a directional analysis," International Journal of Forecasting, Elsevier, vol. 14(3), pages 381-391, September.
- Greer, Mark, 2003. "Directional accuracy tests of long-term interest rate forecasts," International Journal of Forecasting, Elsevier, vol. 19(2), pages 291-298.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Águeda González Abad).
If references are entirely missing, you can add them using this form.