IDEAS home Printed from https://ideas.repec.org/p/boc/bocoec/583.html
   My bibliography  Save this paper

Comparing Density Forecasts via Weighted Likelihood Ratio Tests: Asymptotic and Bootstrap Methods

Author

Listed:
  • Raffaella Giacomini

    (Boston College)

Abstract

This paper proposes tests for comparing the accuracy of density forecasts. The evaluation makes use of scoring rules, which are loss functions defined over the density forecast and the realizations of the variable. In particular, a logarithmic scoring rule leads to the development of asymptotic and bootstrap 'weighted likelihood ratio' tests. I conclude with an application to S&P500 daily returns, comparing the performance of density forecasts obtained from GARCH models with different distributional assumptions.

Suggested Citation

  • Raffaella Giacomini, 2002. "Comparing Density Forecasts via Weighted Likelihood Ratio Tests: Asymptotic and Bootstrap Methods," Boston College Working Papers in Economics 583, Boston College Department of Economics.
    • Handle: RePEc:boc:bocoec:583
    as

    Download full text from publisher

    File URL: http://fmwww.bc.edu/EC-P/wp583.pdf
    File Function: main text
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-1084, September.
    2. White,Halbert, 1996. "Estimation, Inference and Specification Analysis," Cambridge Books, Cambridge University Press, number 9780521574464, January.
    3. Christoffersen, Peter F. & Diebold, Francis X., 1997. "Optimal Prediction Under Asymmetric Loss," Econometric Theory, Cambridge University Press, vol. 13(6), pages 808-817, December.
    4. Francis X. Diebold & Jose A. Lopez, 1995. "Forecast evaluation and combination," Research Paper 9525, Federal Reserve Bank of New York.
    5. Newey, Whitney & West, Kenneth, 2014. "A simple, positive semi-definite, heteroscedasticity and autocorrelation consistent covariance matrix," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 33(1), pages 125-132.
    6. Lopez, Jose A, 2001. "Evaluating the Predictive Accuracy of Volatility Models," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 20(2), pages 87-109, March.
    7. Bollerslev, Tim, 1987. "A Conditionally Heteroskedastic Time Series Model for Speculative Prices and Rates of Return," The Review of Economics and Statistics, MIT Press, vol. 69(3), pages 542-547, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Isao Ishida, 2005. "Scanning Multivariate Conditional Densities with Probability Integral Transforms," CIRJE F-Series CIRJE-F-369, CIRJE, Faculty of Economics, University of Tokyo.
    2. Clements, Michael P. & Franses, Philip Hans & Swanson, Norman R., 2004. "Forecasting economic and financial time-series with non-linear models," International Journal of Forecasting, Elsevier, vol. 20(2), pages 169-183.
    3. James Mitchell & Stephen G. Hall, 2005. "Evaluating, Comparing and Combining Density Forecasts Using the KLIC with an Application to the Bank of England and NIESR ‘Fan’ Charts of Inflation," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 67(s1), pages 995-1033, December.
    4. Torben G. Andersen & Tim Bollerslev & Peter F. Christoffersen & Francis X. Diebold, 2005. "Volatility Forecasting," PIER Working Paper Archive 05-011, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania.
    5. Andersen, Torben G. & Bollerslev, Tim & Christoffersen, Peter F. & Diebold, Francis X., 2006. "Volatility and Correlation Forecasting," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 15, pages 777-878, Elsevier.
    6. Raffaella Giacomini & Halbert White, 2006. "Tests of Conditional Predictive Ability," Econometrica, Econometric Society, vol. 74(6), pages 1545-1578, November.
    7. Valentina Corradi & Norman R. Swanson, 2003. "A Test for Comparing Multiple Misspecified Conditional Distributions," Departmental Working Papers 200314, Rutgers University, Department of Economics.
    8. Corradi, Valentina & Swanson, Norman R., 2006. "Predictive density and conditional confidence interval accuracy tests," Journal of Econometrics, Elsevier, vol. 135(1-2), pages 187-228.
    9. Dick van Dijk & Philip Hans Franses, 2003. "Selecting a Nonlinear Time Series Model using Weighted Tests of Equal Forecast Accuracy," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 65(s1), pages 727-744, December.
    10. Corradi, Valentina & Swanson, Norman R., 2004. "A test for the distributional comparison of simulated and historical data," Economics Letters, Elsevier, vol. 85(2), pages 185-193, November.
    11. Valentina Corradi & Norman Swanson, 2003. "The Block Bootstrap for Parameter Estimation Error In Recursive Estimation Schemes, With Applications to Predictive Evaluation," Departmental Working Papers 200313, Rutgers University, Department of Economics.
    12. Foroni, Claudia & Ravazzolo, Francesco & Rossini, Luca, 2019. "Forecasting daily electricity prices with monthly macroeconomic variables," Working Paper Series 2250, European Central Bank.
    13. Hall, Stephen G. & Mitchell, James, 2007. "Combining density forecasts," International Journal of Forecasting, Elsevier, vol. 23(1), pages 1-13.
    14. Stefania D'Amico, 2005. "Density selection and combination under model ambiguity: an application to stock returns," Finance and Economics Discussion Series 2005-09, Board of Governors of the Federal Reserve System (U.S.).
    15. Stefania D'Amico, 2004. "Density Estimation and Combination under Model Ambiguity," Computing in Economics and Finance 2004 273, Society for Computational Economics.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Raffaella Giacomini & Halbert White, 2006. "Tests of Conditional Predictive Ability," Econometrica, Econometric Society, vol. 74(6), pages 1545-1578, November.
    2. Patton, Andrew J., 2011. "Volatility forecast comparison using imperfect volatility proxies," Journal of Econometrics, Elsevier, vol. 160(1), pages 246-256, January.
    3. Francis X. Diebold & Todd A. Gunther & Anthony S. Tay, "undated". "Evaluating Density Forecasts," CARESS Working Papres 97-18, University of Pennsylvania Center for Analytic Research and Economics in the Social Sciences.
    4. Franses,Philip Hans & Dijk,Dick van & Opschoor,Anne, 2014. "Time Series Models for Business and Economic Forecasting," Cambridge Books, Cambridge University Press, number 9780521520911.
    5. Amisano, Gianni & Giacomini, Raffaella, 2007. "Comparing Density Forecasts via Weighted Likelihood Ratio Tests," Journal of Business & Economic Statistics, American Statistical Association, vol. 25, pages 177-190, April.
    6. Andrew J. Patton & Kevin Sheppard, 2008. "Evaluating Volatility and Correlation Forecasts," OFRC Working Papers Series 2008fe22, Oxford Financial Research Centre.
    7. Torben G. Andersen & Tim Bollerslev & Peter F. Christoffersen & Francis X. Diebold, 2005. "Volatility Forecasting," PIER Working Paper Archive 05-011, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania.
    8. Andersen, Torben G. & Bollerslev, Tim & Christoffersen, Peter F. & Diebold, Francis X., 2006. "Volatility and Correlation Forecasting," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 15, pages 777-878, Elsevier.
    9. Timmermann, Allan & Elliott, Graham & Komunjer, Ivana, 2003. "Estimating Loss Function Parameters," CEPR Discussion Papers 3821, C.E.P.R. Discussion Papers.
    10. Graham Elliott & Ivana Komunjer & Allan Timmermann, 2008. "Biases in Macroeconomic Forecasts: Irrationality or Asymmetric Loss?," Journal of the European Economic Association, MIT Press, vol. 6(1), pages 122-157, March.
    11. Corradi, Valentina & Swanson, Norman R., 2004. "Some recent developments in predictive accuracy testing with nested models and (generic) nonlinear alternatives," International Journal of Forecasting, Elsevier, vol. 20(2), pages 185-199.
    12. Arie Preminger & Uri Ben-zion & David Wettstein, 2007. "The extended switching regression model: allowing for multiple latent state variables," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 26(7), pages 457-473.
    13. Ang, Andrew & Bekaert, Geert & Wei, Min, 2007. "Do macro variables, asset markets, or surveys forecast inflation better?," Journal of Monetary Economics, Elsevier, vol. 54(4), pages 1163-1212, May.
    14. Aaron Smith, 2005. "Forecasting in the presence of level shifts," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 24(8), pages 557-574.
    15. Rossi, Barbara, 2013. "Advances in Forecasting under Instability," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1203-1324, Elsevier.
    16. Diks, Cees & Panchenko, Valentyn & van Dijk, Dick, 2011. "Likelihood-based scoring rules for comparing density forecasts in tails," Journal of Econometrics, Elsevier, vol. 163(2), pages 215-230, August.
    17. Barbara Rossi & Atsushi Inoue, 2012. "Out-of-Sample Forecast Tests Robust to the Choice of Window Size," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 30(3), pages 432-453, April.
    18. Francis X. Diebold & Jose A. Lopez, 1995. "Forecast evaluation and combination," Research Paper 9525, Federal Reserve Bank of New York.
    19. Patton, Andrew J. & Timmermann, Allan, 2005. "Testable implications of forecast optimality," LSE Research Online Documents on Economics 6834, London School of Economics and Political Science, LSE Library.
    20. Jonas Dovern & Hans Manner, 2020. "Order‐invariant tests for proper calibration of multivariate density forecasts," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(4), pages 440-456, June.

    More about this item

    Keywords

    density forecasting; scoring rules; predictive ability; forecast comparison;
    All these keywords.

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:boc:bocoec:583. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Christopher F Baum (email available below). General contact details of provider: https://edirc.repec.org/data/debocus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.