IDEAS home Printed from https://ideas.repec.org/p/bge/wpaper/688.html
   My bibliography  Save this paper

Conditional Predictive Density Evaluation in the Presence of Instabilities

Author

Listed:
  • Barbara Rossi
  • Tatevik Sehkposyan

Abstract

We propose new methods for evaluating predictive densities. The methods include Kolmogorov-Smirnov and Cramer-von Mises-type tests for the correct specification of predictive densities robust to dynamic mis-specification. The novelty is that the tests can detect mis-specification in the predictive densities even if it appears only over a fraction of the sample, due to the presence of instabilities. Our results indicate that our tests are well sized and have good power in detecting mis-specification in predictive densities, even when it is time-varying. An application to density forecasts of the Survey of Professional Forecasters demonstrates the usefulness of the proposed methodologies.

Suggested Citation

  • Barbara Rossi & Tatevik Sehkposyan, 2013. "Conditional Predictive Density Evaluation in the Presence of Instabilities," Working Papers 688, Barcelona Graduate School of Economics.
  • Handle: RePEc:bge:wpaper:688
    as

    Download full text from publisher

    File URL: http://www.barcelonagse.eu/sites/default/files/working_paper_pdfs/688.pdf
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. Diebold, Francis X & Gunther, Todd A & Tay, Anthony S, 1998. "Evaluating Density Forecasts with Applications to Financial Risk Management," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 863-883, November.
    2. Newey, Whitney & West, Kenneth, 2014. "A simple, positive semi-definite, heteroscedasticity and autocorrelation consistent covariance matrix," Applied Econometrics, Publishing House "SINERGIA PRESS", pages 125-132.
    3. Corradi, Valentina & Swanson, Norman R., 2007. "Evaluation of dynamic stochastic general equilibrium models based on distributional comparison of simulated and historical data," Journal of Econometrics, Elsevier, pages 699-723.
    4. Ang, Andrew & Bekaert, Geert & Wei, Min, 2007. "Do macro variables, asset markets, or surveys forecast inflation better?," Journal of Monetary Economics, Elsevier, pages 1163-1212.
    5. Rossi, Barbara, 2005. "Optimal Tests For Nested Model Selection With Underlying Parameter Instability," Econometric Theory, Cambridge University Press, vol. 21(05), pages 962-990, October.
    6. Amisano, Gianni & Giacomini, Raffaella, 2007. "Comparing Density Forecasts via Weighted Likelihood Ratio Tests," Journal of Business & Economic Statistics, American Statistical Association, pages 177-190.
    7. West, Kenneth D & Wilcox, David W, 1996. "A Comparison of Alternative Instrumental Variables Estimators of a Dynamic Linear Model," Journal of Business & Economic Statistics, American Statistical Association, pages 281-293.
    8. Dick Dijk & Philip Hans Franses, 2003. "Selecting a Nonlinear Time Series Model using Weighted Tests of Equal Forecast Accuracy," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 65(s1), pages 727-744, December.
    9. Diks, Cees & Panchenko, Valentyn & van Dijk, Dick, 2011. "Likelihood-based scoring rules for comparing density forecasts in tails," Journal of Econometrics, Elsevier, pages 215-230.
    10. Valentina Corradi & Norman Swanson, 2006. "Predictive Density Evaluation. Revised," Departmental Working Papers 200621, Rutgers University, Department of Economics.
    11. Jushan Bai, 2003. "Testing Parametric Conditional Distributions of Dynamic Models," The Review of Economics and Statistics, MIT Press, pages 531-549.
    12. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-1084, September.
    13. Andrews, Donald W K, 1993. "Tests for Parameter Instability and Structural Change with Unknown Change Point," Econometrica, Econometric Society, pages 821-856.
    14. Corradi, Valentina & Swanson, Norman R., 2006. "Bootstrap conditional distribution tests in the presence of dynamic misspecification," Journal of Econometrics, Elsevier, pages 779-806.
    15. Yongmiao Hong, 2005. "Nonparametric Specification Testing for Continuous-Time Models with Applications to Term Structure of Interest Rates," Review of Financial Studies, Society for Financial Studies, pages 37-84.
    16. Andrade, P. & Ghysels, E. & Idier, J., 2012. "Tails of Inflation Forecasts and Tales of Monetary Policy," Working papers 407, Banque de France.
    17. Corradi, Valentina & Swanson, Norman R., 2006. "Predictive density and conditional confidence interval accuracy tests," Journal of Econometrics, Elsevier, pages 187-228.
    18. Corradi, Valentina & Swanson, Norman R., 2006. "Predictive Density Evaluation," Handbook of Economic Forecasting, Elsevier.
    19. Corradi, Valentina & Swanson, Norman R., 2006. "Predictive density and conditional confidence interval accuracy tests," Journal of Econometrics, Elsevier, pages 187-228.
    20. Inoue, Atsushi, 2001. "Testing For Distributional Change In Time Series," Econometric Theory, Cambridge University Press, vol. 17(01), pages 156-187, February.
    21. repec:hal:journl:peer-00834423 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Davide Delle Monache & Ivan Petrella, 2014. "Adaptive Models and Heavy Tails," Working Papers 720, Queen Mary University of London, School of Economics and Finance.
    2. Barbara Rossi, 2013. "Exchange Rate Predictability," Journal of Economic Literature, American Economic Association, pages 1063-1119.
    3. Barbara Rossi, 2014. "Comment," Journal of Business & Economic Statistics, Taylor & Francis Journals, pages 510-514.
    4. Rossi, Barbara & Sekhposyan, Tatevik, 2014. "Evaluating predictive densities of US output growth and inflation in a large macroeconomic data set," International Journal of Forecasting, Elsevier, pages 662-682.
    5. González-Rivera, Gloria & Sun, Yingying, 2017. "Density forecast evaluation in unstable environments," International Journal of Forecasting, Elsevier, pages 416-432.
    6. Barbara Rossi, 2013. "Exchange Rate Predictability," Journal of Economic Literature, American Economic Association, pages 1063-1119.
    7. Tommaso Proietti & Martyna Marczak & Gianluigi Mazzi, 2017. "Euromind‐ D : A Density Estimate of Monthly Gross Domestic Product for the Euro Area," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 32(3), pages 683-703, April.
    8. Raffaella Giacomini & Barbara Rossi, 2014. "Forecasting in nonstationary environments: What works and what doesn't in reduced-form and structural models," Economics Working Papers 1476, Department of Economics and Business, Universitat Pompeu Fabra.
    9. Gloria Gonzalez-Rivera & Joao Henrique Mazzeu & Esther Ruiz & Helena Veiga, 2017. "A Bootstrap Approach for Generalized Autocontour Testing. Implications for VIX Forecast Densities," Working Papers 201709, University of California at Riverside, Department of Economics.
    10. Raffaella Giacomini & Barbara Rossi, 2015. "Forecasting in Nonstationary Environments: What Works and What Doesn't in Reduced-Form and Structural Models," Annual Review of Economics, Annual Reviews, vol. 7(1), pages 207-229, August.

    More about this item

    Keywords

    predictive density; dynamic mis-specification; instability; structural change; forecast evaluation;

    JEL classification:

    • C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bge:wpaper:688. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Bruno Guallar). General contact details of provider: http://edirc.repec.org/data/bargses.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.