IDEAS home Printed from https://ideas.repec.org/a/jof/jforec/v23y2004i8p541-557.html
   My bibliography  Save this article

Comparing the accuracy of density forecasts from competing models

Author

Listed:
  • Giorgio Valente

    (University of Warwick, UK)

  • Lucio Sarno

    (University of Warwick, UK, and Centre for Economic Policy Research, UK)

Abstract

A rapidly growing literature emphasizes the importance of evaluating the forecast accuracy of empirical models on the basis of density (as opposed to point) forecasting performance. We propose a test statistic for the null hypothesis that two competing models have equal density forecast accuracy. Monte Carlo simulations suggest that the test, which has a known limiting distribution, displays satisfactory size and power properties. The use of the test is illustrated with an application to exchange rate forecasting. Copyright © 2004 John Wiley & Sons, Ltd.

Suggested Citation

  • Giorgio Valente & Lucio Sarno, 2004. "Comparing the accuracy of density forecasts from competing models," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 23(8), pages 541-557.
    • Handle: RePEc:jof:jforec:v:23:y:2004:i:8:p:541-557
    DOI: 10.1002/for.930
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1002/for.930
    File Function: Link to full text; subscription required
    Download Restriction: no
    File URL: https://libkey.io/10.1002/for.930?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Clarida, Richard H. & Sarno, Lucio & Taylor, Mark P. & Valente, Giorgio, 2003. "The out-of-sample success of term structure models as exchange rate predictors: a step beyond," Journal of International Economics, Elsevier, vol. 60(1), pages 61-83, May.
    2. Kenneth Rogoff, 1996. "The Purchasing Power Parity Puzzle," Journal of Economic Literature, American Economic Association, vol. 34(2), pages 647-668, June.
    3. Pagan,Adrian & Ullah,Aman, 1999. "Nonparametric Econometrics," Cambridge Books, Cambridge University Press, number 9780521355643.
    4. Berkowitz, Jeremy, 2001. "Testing Density Forecasts, with Applications to Risk Management," Journal of Business & Economic Statistics, American Statistical Association, vol. 19(4), pages 465-474, October.
    5. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    6. Lucio Sarno & Mark P. Taylor, 2002. "Purchasing Power Parity and the Real Exchange Rate," IMF Staff Papers, Palgrave Macmillan, vol. 49(1), pages 1-5.
    7. Diebold, Francis X & Gunther, Todd A & Tay, Anthony S, 1998. "Evaluating Density Forecasts with Applications to Financial Risk Management," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 863-883, November.
    8. Franses,Philip Hans & Dijk,Dick van, 2000. "Non-Linear Time Series Models in Empirical Finance," Cambridge Books, Cambridge University Press, number 9780521770415.
    9. B. W. Silverman, 1982. "Kernel Density Estimation Using the Fast Fourier Transform," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 31(1), pages 93-99, March.
    10. Lopez, Jose A, 2001. "Evaluating the Predictive Accuracy of Volatility Models," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 20(2), pages 87-109, March.
    11. Wallis, Kenneth F., 2003. "Chi-squared tests of interval and density forecasts, and the Bank of England's fan charts," International Journal of Forecasting, Elsevier, vol. 19(2), pages 165-175.
    12. Anthony Tay & Kenneth F. Wallis, 2000. "Density Forecasting: A Survey," Econometric Society World Congress 2000 Contributed Papers 0370, Econometric Society.
    13. Francis X. Diebold & Jose A. Lopez, 1995. "Forecast evaluation and combination," Research Paper 9525, Federal Reserve Bank of New York.
    14. Fuchun Li & Greg Tkacz, 2001. "A Consistent Bootstrap Test for Conditional Density Functions with Time-Dependent Data," Staff Working Papers 01-21, Bank of Canada.
    15. Hendry, David F., 1984. "Monte carlo experimentation in econometrics," Handbook of Econometrics, in: Z. Griliches† & M. D. Intriligator (ed.), Handbook of Econometrics, edition 1, volume 2, chapter 16, pages 937-976, Elsevier.
    16. Corker, R. J. & Holly, S. & Ellis, R. G., 1986. "Uncertainty and forecast precision," International Journal of Forecasting, Elsevier, vol. 2(1), pages 53-53.
    17. Anderson, N. H. & Hall, P. & Titterington, D. M., 1994. "Two-Sample Test Statistics for Measuring Discrepancies Between Two Multivariate Probability Density Functions Using Kernel-Based Density Estimates," Journal of Multivariate Analysis, Elsevier, vol. 50(1), pages 41-54, July.
    18. Richard H. Clarida & Mark P. Taylor, 1997. "The Term Structure Of Forward Exchange Premiums And The Forecastability Of Spot Exchange Rates: Correcting The Errors," The Review of Economics and Statistics, MIT Press, vol. 79(3), pages 353-361, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Hua, Zhongsheng & Zhang, Bin, 2008. "Improving density forecast by modeling asymmetric features: An application to S&P500 returns," European Journal of Operational Research, Elsevier, vol. 185(2), pages 716-725, March.
    2. Erik Kole & Dick Dijk, 2017. "How to Identify and Forecast Bull and Bear Markets?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 32(1), pages 120-139, January.
    3. Andreas Lindemann & Christian Dunis & Paulo Lisboa, 2005. "Probability distributions and leveraged trading strategies: an application of Gaussian mixture models to the Morgan Stanley Technology Index Tracking Fund," Quantitative Finance, Taylor & Francis Journals, vol. 5(5), pages 459-474.
    4. Mouratidis, Kostas, 2008. "Evaluating currency crises: A Bayesian Markov switching approach," Journal of Macroeconomics, Elsevier, vol. 30(4), pages 1688-1711, December.
    5. Cees Diks & Valentyn Panchenko & Dick van Dijk, 2008. "Partial Likelihood-Based Scoring Rules for Evaluating Density Forecasts in Tails," Tinbergen Institute Discussion Papers 08-050/4, Tinbergen Institute.
    6. Lucio Sarno & Giorgio Valente, 2005. "Modelling and forecasting stock returns: exploiting the futures market, regime shifts and international spillovers," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 20(3), pages 345-376, March.
    7. John Stachurski & Vance Martin, 2008. "Computing the Distributions of Economic Models via Simulation," Econometrica, Econometric Society, vol. 76(2), pages 443-450, March.
    8. Sarno, Lucio & Valente, Giorgio, 2005. "Empirical exchange rate models and currency risk: some evidence from density forecasts," Journal of International Money and Finance, Elsevier, vol. 24(2), pages 363-385, March.
    9. Diks, Cees & Fang, Hao, 2020. "Comparing density forecasts in a risk management context," International Journal of Forecasting, Elsevier, vol. 36(2), pages 531-551.
    10. Clarida, Richard H. & Sarno, Lucio & Taylor, Mark P. & Valente, Giorgio, 2003. "The out-of-sample success of term structure models as exchange rate predictors: a step beyond," Journal of International Economics, Elsevier, vol. 60(1), pages 61-83, May.
    11. Diks, Cees & Panchenko, Valentyn & van Dijk, Dick, 2011. "Likelihood-based scoring rules for comparing density forecasts in tails," Journal of Econometrics, Elsevier, vol. 163(2), pages 215-230, August.
    12. Lucio Sarno, 2003. "Nonlinear Exchange Rate Models: A Selective Overview," Rivista di Politica Economica, SIPI Spa, vol. 93(4), pages 3-46, July-Augu.
    13. Tsyplakov, Alexander, 2014. "Theoretical guidelines for a partially informed forecast examiner," MPRA Paper 55017, University Library of Munich, Germany.
    14. repec:hal:journl:peer-00834423 is not listed on IDEAS
    15. Tae-Hwy Lee & Yong Bao & Burak Saltoğlu, 2007. "Comparing density forecast models Previous versions of this paper have been circulated with the title, 'A Test for Density Forecast Comparison with Applications to Risk Management' since October 2003;," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 26(3), pages 203-225.
    16. Andrea Cipollini & Bassam Fattouh & Kostas Mouratidis, 2009. "Fiscal Readjustments In The United States: A Nonlinear Time‐Series Analysis," Economic Inquiry, Western Economic Association International, vol. 47(1), pages 34-54, January.
    17. Shamiri, Ahmed & Shaari, Abu Hassan & Isa, Zaidi, 2008. "Comparing the accuracy of density forecasts from competing GARCH models," MPRA Paper 13662, University Library of Munich, Germany.
    18. John Galbraith & Greg Tkacz, 2007. "How Far Can Forecasting Models Forecast? Forecast Content Horizons for Some Important Macroeconomic Variables," Staff Working Papers 07-1, Bank of Canada.
    19. Anthony Garratt & Kevin Lee, 2006. "Investing Under Model Uncertainty: Decision Based Evaluation of Exchange Rate and Interest Rate Forecasts in the US, UK and Japan," Birkbeck Working Papers in Economics and Finance 0616, Birkbeck, Department of Economics, Mathematics & Statistics.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lucio Sarno, 2003. "Nonlinear Exchange Rate Models: A Selective Overview," Rivista di Politica Economica, SIPI Spa, vol. 93(4), pages 3-46, July-Augu.
    2. Polanski, Arnold & Stoja, Evarist, 2012. "Efficient evaluation of multidimensional time-varying density forecasts, with applications to risk management," International Journal of Forecasting, Elsevier, vol. 28(2), pages 343-352.
    3. Lucio Sarno, 2005. "Viewpoint: Towards a solution to the puzzles in exchange rate economics: where do we stand?," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 38(3), pages 673-708, August.
    4. Clarida, Richard H. & Sarno, Lucio & Taylor, Mark P. & Valente, Giorgio, 2003. "The out-of-sample success of term structure models as exchange rate predictors: a step beyond," Journal of International Economics, Elsevier, vol. 60(1), pages 61-83, May.
    5. Sarno, Lucio & Valente, Giorgio, 2005. "Empirical exchange rate models and currency risk: some evidence from density forecasts," Journal of International Money and Finance, Elsevier, vol. 24(2), pages 363-385, March.
    6. Richard H. Clarida & Lucio Sarno & Mark P. Taylor & Giorgio Valente, 2006. "The Role of Asymmetries and Regime Shifts in the Term Structure of Interest Rates," The Journal of Business, University of Chicago Press, vol. 79(3), pages 1193-1224, May.
    7. Rapach, David E. & Wohar, Mark E., 2006. "The out-of-sample forecasting performance of nonlinear models of real exchange rate behavior," International Journal of Forecasting, Elsevier, vol. 22(2), pages 341-361.
    8. Weron, Rafał, 2014. "Electricity price forecasting: A review of the state-of-the-art with a look into the future," International Journal of Forecasting, Elsevier, vol. 30(4), pages 1030-1081.
    9. Federico Bassetti & Roberto Casarin & Francesco Ravazzolo, 2019. "Density Forecasting," BEMPS - Bozen Economics & Management Paper Series BEMPS59, Faculty of Economics and Management at the Free University of Bozen.
    10. Torben G. Andersen & Tim Bollerslev & Peter F. Christoffersen & Francis X. Diebold, 2005. "Volatility Forecasting," PIER Working Paper Archive 05-011, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania.
    11. Andersen, Torben G. & Bollerslev, Tim & Christoffersen, Peter F. & Diebold, Francis X., 2006. "Volatility and Correlation Forecasting," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 15, pages 777-878, Elsevier.
    12. Barbara Rossi, 2013. "Exchange Rate Predictability," Journal of Economic Literature, American Economic Association, vol. 51(4), pages 1063-1119, December.
    13. Mehmet Balcilar & Rangan Gupta & Stephen M. Miller, 2015. "The out-of-sample forecasting performance of nonlinear models of regional housing prices in the US," Applied Economics, Taylor & Francis Journals, vol. 47(22), pages 2259-2277, May.
    14. Hall, Stephen G. & Mitchell, James, 2007. "Combining density forecasts," International Journal of Forecasting, Elsevier, vol. 23(1), pages 1-13.
    15. Terasvirta, Timo, 2006. "Forecasting economic variables with nonlinear models," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 8, pages 413-457, Elsevier.
    16. Goodness C. Aye & Mehmet Balcilar & Adél Bosch & Rangan Gupta & Francois Stofberg, 2013. "The out-of-sample forecasting performance of non-linear models of real exchange rate behaviour: The case of the South African Rand," European Journal of Comparative Economics, Cattaneo University (LIUC), vol. 10(1), pages 121-148, April.
    17. Kostas Mouratidis & Dimitris Kenourgios & Aris Samitas, 2010. "Evaluating currency crisis:A multivariate Markov switching approach," Working Papers 2010018, The University of Sheffield, Department of Economics, revised Oct 2010.
    18. Li, Yushu & Andersson, Jonas, 2014. "A Likelihood Ratio and Markov Chain Based Method to Evaluate Density Forecasting," Discussion Papers 2014/12, Norwegian School of Economics, Department of Business and Management Science.
    19. Knut Are Aastveit & Claudia Foroni & Francesco Ravazzolo, 2017. "Density Forecasts With Midas Models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 32(4), pages 783-801, June.
    20. Cees Diks & Valentyn Panchenko & Dick van Dijk, 2008. "Partial Likelihood-Based Scoring Rules for Evaluating Density Forecasts in Tails," Tinbergen Institute Discussion Papers 08-050/4, Tinbergen Institute.

    More about this item

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • F31 - International Economics - - International Finance - - - Foreign Exchange

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:jof:jforec:v:23:y:2004:i:8:p:541-557. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley-Blackwell Digital Licensing or Christopher F. Baum (email available below). General contact details of provider: http://www3.interscience.wiley.com/cgi-bin/jhome/2966 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.