A statistical test for forecast evaluation under a discrete loss function
AbstractWe propose a new approach to evaluating the usefulness of a set of forecasts, based on the use of a discrete loss function de ned on the space of data and forecasts. Exist- ing procedures for such an evaluation either do not allow for formal testing, or use tests statistics based just on the frequency distribution of (data , forecasts)-pairs. They can easily lead to misleading conclusions in some reasonable situations, because of the way they formalize the underlying null hypothesis that the set of forecasts is not useful. Even though the ambiguity of the underlying null hypothesis precludes us from per- forming a standard analysis of the size and power of the tests, we get results suggesting that the proposed DISC test performs better than its competitors.
Download InfoIf you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
Bibliographic InfoPaper provided by Universidad Complutense de Madrid, Facultad de Ciencias Económicas y Empresariales, Instituto Complutense de Análisis Económico in its series Documentos de Trabajo del ICAE with number 2011-07.
Length: 17 pages
Date of creation: 2011
Date of revision:
Postal: Facultad de Ciencias Económicas y Empresariales. Pabellón prefabricado, 1ª Planta, ala norte. Campus de Somosaguas, 28223 - POZUELO DE ALARCÓN (MADRID)
Other versions of this item:
- Francisco Javier Eransus & Alfonso Novales Cinca, 2014. "A statistical test for forecast evaluation under a discrete loss function," Documentos de Trabajo del ICAE 2014-24, Universidad Complutense de Madrid, Facultad de Ciencias Económicas y Empresariales, Instituto Complutense de Análisis Económico.
- C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
- C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
- C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
This paper has been announced in the following NEP Reports:
- NEP-ALL-2011-04-23 (All new papers)
- NEP-ECM-2011-04-23 (Econometrics)
- NEP-ETS-2011-04-23 (Econometric Time Series)
- NEP-FOR-2011-04-23 (Forecasting)
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Mills, Terence C. & Pepper, Gordon T., 1999. "Assessing the forecasters: an analysis of the forecasting records of the Treasury, the London Business School and the National Institute," International Journal of Forecasting, Elsevier, vol. 15(3), pages 247-257, July.
- Pesaran, M.H. & Timmermann, A., 1990.
"A Simple Non-Parametric Test Of Predictive Performance,"
29, California Los Angeles - Applied Econometrics.
- Pesaran, M Hashem & Timmermann, Allan, 1992. "A Simple Nonparametric Test of Predictive Performance," Journal of Business & Economic Statistics, American Statistical Association, vol. 10(4), pages 561-65, October.
- Pesaran, M.H. & Timmermann, A., 1990. "A Simple, Non-Parametric Test Of Predictive Performance," Cambridge Working Papers in Economics 9021, Faculty of Economics, University of Cambridge.
- Joutz, Fred & Stekler, H. O., 2000. "An evaluation of the predictions of the Federal Reserve," International Journal of Forecasting, Elsevier, vol. 16(1), pages 17-38.
- Ash, J. C. K. & Smyth, D. J. & Heravi, S. M., 1998. "Are OECD forecasts rational and useful?: a directional analysis," International Journal of Forecasting, Elsevier, vol. 14(3), pages 381-391, September.
- Merton, Robert C, 1981. "On Market Timing and Investment Performance. I. An Equilibrium Theory of Value for Market Forecasts," The Journal of Business, University of Chicago Press, vol. 54(3), pages 363-406, July.
- Henriksson, Roy D & Merton, Robert C, 1981. "On Market Timing and Investment Performance. II. Statistical Procedures for Evaluating Forecasting Skills," The Journal of Business, University of Chicago Press, vol. 54(4), pages 513-33, October.
- Greer, Mark, 2003. "Directional accuracy tests of long-term interest rate forecasts," International Journal of Forecasting, Elsevier, vol. 19(2), pages 291-298.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Águeda González Abad).
If references are entirely missing, you can add them using this form.