IDEAS home Printed from https://ideas.repec.org/a/eee/econom/v208y2019i2p638-657.html
   My bibliography  Save this article

Alternative tests for correct specification of conditional predictive densities

Author

Listed:
  • Rossi, Barbara
  • Sekhposyan, Tatevik

Abstract

We propose a new framework for evaluating predictive densities in an environment where the estimation error of the parameters used to construct the densities is preserved asymptotically under the null hypothesis. The tests offer a simple way to evaluate the correct specification of predictive densities, where both the model specification and its estimation technique are evaluated jointly. Monte Carlo simulation results indicate that our tests are well sized and have good power in detecting misspecification. An empirical application to density forecasts of the Survey of Professional Forecasters shows the usefulness of our methodology.

Suggested Citation

  • Rossi, Barbara & Sekhposyan, Tatevik, 2019. "Alternative tests for correct specification of conditional predictive densities," Journal of Econometrics, Elsevier, vol. 208(2), pages 638-657.
    • Handle: RePEc:eee:econom:v:208:y:2019:i:2:p:638-657
    DOI: 10.1016/j.jeconom.2018.07.008
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0304407618302197
    Download Restriction: Full text for ScienceDirect subscribers only

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Billio, Monica & Casarin, Roberto & Ravazzolo, Francesco & van Dijk, Herman K., 2013. "Time-varying combinations of predictive densities using nonlinear filtering," Journal of Econometrics, Elsevier, vol. 177(2), pages 213-232.
    2. Raffaella Giacomini & Halbert White, 2006. "Tests of Conditional Predictive Ability," Econometrica, Econometric Society, vol. 74(6), pages 1545-1578, November.
    3. González-Rivera, Gloria & Senyuz, Zeynep & Yoldas, Emre, 2011. "Autocontours: Dynamic Specification Testing," Journal of Business & Economic Statistics, American Statistical Association, vol. 29(1), pages 186-200.
    4. Barbara Rossi & Tatevik Sekhposyan, 2016. "Forecast Rationality Tests in the Presence of Instabilities, with Applications to Federal Reserve and Survey Forecasts," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 31(3), pages 507-532, April.
    5. Malte Knüppel, 2015. "Evaluating the Calibration of Multi-Step-Ahead Density Forecasts Using Raw Moments," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 33(2), pages 270-281, April.
    6. Christian Bontemps & Nour Meddahi, 2012. "Testing distributional assumptions: A GMM aproach," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 27(6), pages 978-1012, September.
    7. Berkowitz, Jeremy, 2001. "Testing Density Forecasts, with Applications to Risk Management," Journal of Business & Economic Statistics, American Statistical Association, vol. 19(4), pages 465-474, October.
    8. Davidson, James, 1994. "Stochastic Limit Theory: An Introduction for Econometricians," OUP Catalogue, Oxford University Press, number 9780198774037.
    9. Croushore, Dean & Stark, Tom, 2001. "A real-time data set for macroeconomists," Journal of Econometrics, Elsevier, vol. 105(1), pages 111-130, November.
    10. Corradi, Valentina & Swanson, Norman R., 2006. "Bootstrap conditional distribution tests in the presence of dynamic misspecification," Journal of Econometrics, Elsevier, vol. 133(2), pages 779-806, August.
    11. Yongmiao Hong, 2005. "Nonparametric Specification Testing for Continuous-Time Models with Applications to Term Structure of Interest Rates," Review of Financial Studies, Society for Financial Studies, vol. 18(1), pages 37-84.
    12. Maik H. Wolters, 2015. "Evaluating Point and Density Forecasts of DSGE Models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 30(1), pages 74-96, January.
    13. Michael P. Clements, 2004. "Evaluating the Bank of England Density Forecasts of Inflation," Economic Journal, Royal Economic Society, vol. 114(498), pages 844-866, October.
    14. Wolters, Maik H., 2011. "Forecasting under Model Uncertainty," VfS Annual Conference 2011 (Frankfurt, Main): The Order of the World Economy - Lessons from the Crisis 48723, Verein für Socialpolitik / German Economic Association.
    15. Graham Elliott & Allan Timmermann, 2016. "Economic Forecasting," Economics Books, Princeton University Press, edition 1, number 10740, October.
    16. Inoue, Atsushi, 2001. "Testing For Distributional Change In Time Series," Econometric Theory, Cambridge University Press, vol. 17(1), pages 156-187, February.
    17. González-Rivera, Gloria & Sun, Yingying, 2015. "Generalized autocontours: Evaluation of multivariate density models," International Journal of Forecasting, Elsevier, vol. 31(3), pages 799-814.
    18. Jushan Bai & Serena Ng, 2005. "Tests for Skewness, Kurtosis, and Normality for Time Series Data," Journal of Business & Economic Statistics, American Statistical Association, vol. 23, pages 49-60, January.
    19. Diebold, Francis X & Gunther, Todd A & Tay, Anthony S, 1998. "Evaluating Density Forecasts with Applications to Financial Risk Management," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 863-883, November.
    20. González-Rivera, Gloria & Yoldas, Emre, 2012. "Autocontour-based evaluation of multivariate predictive densities," International Journal of Forecasting, Elsevier, vol. 28(2), pages 328-342.
    21. Stark, Tom & Croushore, Dean, 2002. "Forecasting with a real-time data set for macroeconomists," Journal of Macroeconomics, Elsevier, vol. 24(4), pages 507-531, December.
    22. Amisano, Gianni & Giacomini, Raffaella, 2007. "Comparing Density Forecasts via Weighted Likelihood Ratio Tests," Journal of Business & Economic Statistics, American Statistical Association, vol. 25, pages 177-190, April.
    23. Wolden Bache, Ida & Sofie Jore, Anne & Mitchell, James & Vahey, Shaun P., 2011. "Combining VAR and DSGE forecast densities," Journal of Economic Dynamics and Control, Elsevier, vol. 35(10), pages 1659-1670, October.
    24. Hong, Yongmiao & Li, Haitao & Zhao, Feng, 2007. "Can the random walk model be beaten in out-of-sample density forecasts? Evidence from intraday foreign exchange rates," Journal of Econometrics, Elsevier, vol. 141(2), pages 736-776, December.
    25. Valentina Corradi & Norman Swanson, 2006. "Predictive Density Evaluation. Revised," Departmental Working Papers 200621, Rutgers University, Department of Economics.
    26. Gürkaynak, Refet S. & Kisacikoglu, Burçin & Rossi, Barbara, 2013. "Do DSGE Models Forecast More Accurately Out-of-Sample than VAR Models?," CEPR Discussion Papers 9576, C.E.P.R. Discussion Papers.
    27. Rochelle M. Edge & Michael T. Kiley & Jean-Philippe Laforte, 2010. "A comparison of forecast performance between Federal Reserve staff forecasts, simple reduced-form models, and a DSGE model," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(4), pages 720-754.
    28. Barbara Rossi & Atsushi Inoue, 2012. "Out-of-Sample Forecast Tests Robust to the Choice of Window Size," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 30(3), pages 432-453, April.
    29. Knut Are Aastveit & Claudia Foroni & Francesco Ravazzolo, 2017. "Density Forecasts With Midas Models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 32(4), pages 783-801, June.
    30. Corradi, Valentina & Swanson, Norman R., 2006. "Predictive Density Evaluation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 5, pages 197-284, Elsevier.
    31. Ang, Andrew & Bekaert, Geert & Wei, Min, 2007. "Do macro variables, asset markets, or surveys forecast inflation better?," Journal of Monetary Economics, Elsevier, vol. 54(4), pages 1163-1212, May.
    32. Dick van Dijk & Philip Hans Franses, 2003. "Selecting a Nonlinear Time Series Model using Weighted Tests of Equal Forecast Accuracy," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 65(s1), pages 727-744, December.
    33. Diks, Cees & Panchenko, Valentyn & van Dijk, Dick, 2011. "Likelihood-based scoring rules for comparing density forecasts in tails," Journal of Econometrics, Elsevier, vol. 163(2), pages 215-230, August.
    34. Ravazzolo Francesco & Vahey Shaun P., 2014. "Forecast densities for economic aggregates from disaggregate ensembles," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 18(4), pages 1-15, September.
    35. Corradi, Valentina & Swanson, Norman R., 2007. "Evaluation of dynamic stochastic general equilibrium models based on distributional comparison of simulated and historical data," Journal of Econometrics, Elsevier, vol. 136(2), pages 699-723, February.
    36. Frank Smets & Rafael Wouters, 2007. "Shocks and Frictions in US Business Cycles: A Bayesian DSGE Approach," American Economic Review, American Economic Association, vol. 97(3), pages 586-606, June.
    37. Warne, Anders & Coenen, Günter & Christoffel, Kai, 2010. "Forecasting with DSGE models," Working Paper Series 1185, European Central Bank.
    38. James H. Stock & Mark W. Watson, 2007. "Why Has U.S. Inflation Become Harder to Forecast?," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 39(s1), pages 3-33, February.
    39. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
    40. Newey, Whitney & West, Kenneth, 2014. "A simple, positive semi-definite, heteroscedasticity and autocorrelation consistent covariance matrix," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 33(1), pages 125-132.
    41. N/A, 2016. "The World Economy: Forecast Summary," National Institute Economic Review, National Institute of Economic and Social Research, vol. 238(1), pages 2-2, November.
    42. Jushan Bai, 2003. "Testing Parametric Conditional Distributions of Dynamic Models," The Review of Economics and Statistics, MIT Press, vol. 85(3), pages 531-549, August.
    43. Rochelle M. Edge & Refet S. Gurkaynak, 2010. "How Useful Are Estimated DSGE Model Forecasts for Central Bankers?," Brookings Papers on Economic Activity, Economic Studies Program, The Brookings Institution, vol. 41(2 (Fall)), pages 209-259.
    44. repec:hal:journl:peer-00834423 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gergely Ganics & Barbara Rossi & Tatevik Sekhposyan, 2019. "From fixed-event to fixed-horizon density forecasts: obtaining measures of multi-horizon uncertainty from survey density forecasts," Working Papers 1947, Banco de España;Working Papers Homepage.
    2. Richard K. Crump & Domenico Giannone & Sean Hundtofte, 2018. "Changing Risk-Return Profiles," Liberty Street Economics 20181004, Federal Reserve Bank of New York.
    3. Matteo Iacopini & Francesco Ravazzolo & Luca Rossini, 2020. "Proper scoring rules for evaluating asymmetry in density forecasting," Papers 2006.11265, arXiv.org, revised Sep 2020.
    4. Plakandaras, Vasilios & Gogas, Periklis & Papadimitriou, Theophilos & Gupta, Rangan, 2019. "A re-evaluation of the term spread as a leading indicator," International Review of Economics & Finance, Elsevier, vol. 64(C), pages 476-492.
    5. Adams, Patrick & Adrian, Tobias & Boyarchenko, Nina & Giannone, Domenico, 2020. "Forecasting Macroeconomic Risks," CEPR Discussion Papers 14436, C.E.P.R. Discussion Papers.
    6. Tobias Adrian & Nina Boyarchenko & Domenico Giannone, 2019. "Vulnerable Growth," American Economic Review, American Economic Association, vol. 109(4), pages 1263-1289, April.
    7. Michael W. McCracken, 2020. "Diverging Tests of Equal Predictive Ability," Econometrica, Econometric Society, vol. 88(4), pages 1753-1754, July.
    8. Taylor, James W., 2020. "A strategic predictive distribution for tests of probabilistic calibration," International Journal of Forecasting, Elsevier, vol. 36(4), pages 1380-1388.
    9. Marcus P. A. Cobb, 2020. "Aggregate density forecasting from disaggregate components using Bayesian VARs," Empirical Economics, Springer, vol. 58(1), pages 287-312, January.
    10. Alex Tagliabracci, 2020. "Asymmetry in the conditional distribution of euro-area inflation," Temi di discussione (Economic working papers) 1270, Bank of Italy, Economic Research and International Relations Area.
    11. Niango Ange Joseph Yapi, 2020. "Exchange rate predictive densities and currency risks: A quantile regression approach," EconomiX Working Papers 2020-16, University of Paris Nanterre, EconomiX.
    12. Wojciech Charemza & Carlos Diaz Vela & Svetlana Makarova, 2013. "Too many skew normal distributions? The practitioner’s perspective," Discussion Papers in Economics 13/07, Division of Economics, School of Business, University of Leicester.
    13. Jonas Dovern & Hans Manner, 2020. "Order‐invariant tests for proper calibration of multivariate density forecasts," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(4), pages 440-456, June.
    14. Fabio Busetti & Michele Caivano & Davide Delle Monache & Claudia Pacella, 2020. "The time-varying risk of Italian GDP," Temi di discussione (Economic working papers) 1288, Bank of Italy, Economic Research and International Relations Area.
    15. Barbara Rossi, 2019. "Forecasting in the Presence of Instabilities: How Do We Know Whether Models Predict Well and How to Improve Them," Working Papers 1162, Barcelona Graduate School of Economics.
    16. J. David López-Salido & Francesca Loria, 2020. "Inflation at Risk," Finance and Economics Discussion Series 2020-013, Board of Governors of the Federal Reserve System (U.S.).
    17. López-Salido, J David & Loria, Francesca, 2019. "Inflation at Risk," CEPR Discussion Papers 14074, C.E.P.R. Discussion Papers.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Rossi, Barbara, 2013. "Advances in Forecasting under Instability," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1203-1324, Elsevier.
    2. Rossi, Barbara & Sekhposyan, Tatevik, 2013. "Conditional predictive density evaluation in the presence of instabilities," Journal of Econometrics, Elsevier, vol. 177(2), pages 199-212.
    3. Diks, Cees & Panchenko, Valentyn & van Dijk, Dick, 2011. "Likelihood-based scoring rules for comparing density forecasts in tails," Journal of Econometrics, Elsevier, vol. 163(2), pages 215-230, August.
    4. Barbara Rossi, 2019. "Forecasting in the presence of instabilities: How do we know whether models predict well and how to improve them," Economics Working Papers 1711, Department of Economics and Business, Universitat Pompeu Fabra.
    5. Jonas Dovern & Hans Manner, 2020. "Order‐invariant tests for proper calibration of multivariate density forecasts," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(4), pages 440-456, June.
    6. Gaglianone, Wagner Piazza & Marins, Jaqueline Terra Moura, 2017. "Evaluation of exchange rate point and density forecasts: An application to Brazil," International Journal of Forecasting, Elsevier, vol. 33(3), pages 707-728.
    7. Raffaella Giacomini & Barbara Rossi, 2015. "Forecasting in Nonstationary Environments: What Works and What Doesn't in Reduced-Form and Structural Models," Annual Review of Economics, Annual Reviews, vol. 7(1), pages 207-229, August.
    8. Gelain, Paolo & Iskrev, Nikolay & J. Lansing, Kevin & Mendicino, Caterina, 2019. "Inflation dynamics and adaptive expectations in an estimated DSGE model," Journal of Macroeconomics, Elsevier, vol. 59(C), pages 258-277.
    9. Clements, Michael P., 2018. "Are macroeconomic density forecasts informative?," International Journal of Forecasting, Elsevier, vol. 34(2), pages 181-198.
    10. Knut Are Aastveit & Claudia Foroni & Francesco Ravazzolo, 2017. "Density Forecasts With Midas Models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 32(4), pages 783-801, June.
    11. Cees Diks & Valentyn Panchenko & Dick van Dijk, 2008. "Partial Likelihood-Based Scoring Rules for Evaluating Density Forecasts in Tails," Tinbergen Institute Discussion Papers 08-050/4, Tinbergen Institute.
    12. Rodríguez, Gabriel, 2017. "Modeling Latin-American stock and Forex markets volatility: Empirical application of a model with random level shifts and genuine long memory," The North American Journal of Economics and Finance, Elsevier, vol. 42(C), pages 393-420.
    13. Anne Sofie Jore & James Mitchell & Shaun P. Vahey, 2010. "Combining forecast densities from VARs with uncertain instabilities," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(4), pages 621-634.
    14. Veiga, Helena & Ruiz, Esther & González-Rivera, Gloria & Gonçalves Mazzeu, Joao Henrique, 2016. "A Bootstrap Approach for Generalized Autocontour Testing," DES - Working Papers. Statistics and Econometrics. WS 23457, Universidad Carlos III de Madrid. Departamento de Estadística.
    15. Bekiros Stelios & Paccagnini Alessia, 2015. "Estimating point and density forecasts for the US economy with a factor-augmented vector autoregressive DSGE model," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 19(2), pages 107-136, April.
    16. Garratt, Anthony & Mitchell, James & Vahey, Shaun P., 2014. "Measuring output gap nowcast uncertainty," International Journal of Forecasting, Elsevier, vol. 30(2), pages 268-279.
    17. Cardani, Roberta & Paccagnini, Alessia & Villa, Stefania, 2019. "Forecasting with instabilities: An application to DSGE models with financial frictions," Journal of Macroeconomics, Elsevier, vol. 61(C), pages 1-1.
    18. McAdam, Peter & Warne, Anders, 2020. "Density forecast combinations: the real-time dimension," Working Paper Series 2378, European Central Bank.
    19. Dovern, Jonas & Manner, Hans, 2016. "Robust Evaluation of Multivariate Density Forecasts," VfS Annual Conference 2016 (Augsburg): Demographic Change 145547, Verein für Socialpolitik / German Economic Association.
    20. Dovern, Jonas & Manner, Hans, 2016. "Order Invariant Evaluation of Multivariate Density Forecasts," Working Papers 0608, University of Heidelberg, Department of Economics.

    More about this item

    Keywords

    Predictive density; Probability integral transform; Kolmogorov–Smirnov test; Cramér–von Mises test; Forecast evaluation;
    All these keywords.

    JEL classification:

    • C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:econom:v:208:y:2019:i:2:p:638-657. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Haili He). General contact details of provider: http://www.elsevier.com/locate/jeconom .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.