IDEAS home Printed from https://ideas.repec.org/a/eee/econom/v208y2019i2p638-657.html
   My bibliography  Save this article

Alternative tests for correct specification of conditional predictive densities

Author

Listed:
  • Rossi, Barbara
  • Sekhposyan, Tatevik

Abstract

We propose a new framework for evaluating predictive densities in an environment where the estimation error of the parameters used to construct the densities is preserved asymptotically under the null hypothesis. The tests offer a simple way to evaluate the correct specification of predictive densities, where both the model specification and its estimation technique are evaluated jointly. Monte Carlo simulation results indicate that our tests are well sized and have good power in detecting misspecification. An empirical application to density forecasts of the Survey of Professional Forecasters shows the usefulness of our methodology.

Suggested Citation

  • Rossi, Barbara & Sekhposyan, Tatevik, 2019. "Alternative tests for correct specification of conditional predictive densities," Journal of Econometrics, Elsevier, vol. 208(2), pages 638-657.
    • Handle: RePEc:eee:econom:v:208:y:2019:i:2:p:638-657
    DOI: 10.1016/j.jeconom.2018.07.008
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0304407618302197
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jeconom.2018.07.008?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Ravazzolo Francesco & Vahey Shaun P., 2014. "Forecast densities for economic aggregates from disaggregate ensembles," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 18(4), pages 1-15, September.
    2. Barbara Rossi & Atsushi Inoue, 2012. "Out-of-Sample Forecast Tests Robust to the Choice of Window Size," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 30(3), pages 432-453, April.
    3. Billio, Monica & Casarin, Roberto & Ravazzolo, Francesco & van Dijk, Herman K., 2013. "Time-varying combinations of predictive densities using nonlinear filtering," Journal of Econometrics, Elsevier, vol. 177(2), pages 213-232.
    4. Corradi, Valentina & Swanson, Norman R., 2007. "Evaluation of dynamic stochastic general equilibrium models based on distributional comparison of simulated and historical data," Journal of Econometrics, Elsevier, vol. 136(2), pages 699-723, February.
    5. Raffaella Giacomini & Halbert White, 2006. "Tests of Conditional Predictive Ability," Econometrica, Econometric Society, vol. 74(6), pages 1545-1578, November.
    6. González-Rivera, Gloria & Senyuz, Zeynep & Yoldas, Emre, 2011. "Autocontours: Dynamic Specification Testing," Journal of Business & Economic Statistics, American Statistical Association, vol. 29(1), pages 186-200.
    7. Barbara Rossi & Tatevik Sekhposyan, 2016. "Forecast Rationality Tests in the Presence of Instabilities, with Applications to Federal Reserve and Survey Forecasts," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 31(3), pages 507-532, April.
    8. Malte Knüppel, 2015. "Evaluating the Calibration of Multi-Step-Ahead Density Forecasts Using Raw Moments," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 33(2), pages 270-281, April.
    9. Frank Smets & Rafael Wouters, 2007. "Shocks and Frictions in US Business Cycles: A Bayesian DSGE Approach," American Economic Review, American Economic Association, vol. 97(3), pages 586-606, June.
    10. Dick van Dijk & Philip Hans Franses, 2003. "Selecting a Nonlinear Time Series Model using Weighted Tests of Equal Forecast Accuracy," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 65(s1), pages 727-744, December.
    11. Christian Bontemps & Nour Meddahi, 2012. "Testing distributional assumptions: A GMM aproach," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 27(6), pages 978-1012, September.
    12. Stark, Tom & Croushore, Dean, 2002. "Forecasting with a real-time data set for macroeconomists," Journal of Macroeconomics, Elsevier, vol. 24(4), pages 507-531, December.
    13. Berkowitz, Jeremy, 2001. "Testing Density Forecasts, with Applications to Risk Management," Journal of Business & Economic Statistics, American Statistical Association, vol. 19(4), pages 465-474, October.
    14. Davidson, James, 1994. "Stochastic Limit Theory: An Introduction for Econometricians," OUP Catalogue, Oxford University Press, number 9780198774037.
    15. Croushore, Dean & Stark, Tom, 2001. "A real-time data set for macroeconomists," Journal of Econometrics, Elsevier, vol. 105(1), pages 111-130, November.
    16. Corradi, Valentina & Swanson, Norman R., 2006. "Bootstrap conditional distribution tests in the presence of dynamic misspecification," Journal of Econometrics, Elsevier, vol. 133(2), pages 779-806, August.
    17. Warne, Anders & Coenen, Günter & Christoffel, Kai, 2010. "Forecasting with DSGE models," Working Paper Series 1185, European Central Bank.
    18. Ang, Andrew & Bekaert, Geert & Wei, Min, 2007. "Do macro variables, asset markets, or surveys forecast inflation better?," Journal of Monetary Economics, Elsevier, vol. 54(4), pages 1163-1212, May.
    19. Knut Are Aastveit & Claudia Foroni & Francesco Ravazzolo, 2017. "Density Forecasts With Midas Models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 32(4), pages 783-801, June.
    20. Yongmiao Hong, 2005. "Nonparametric Specification Testing for Continuous-Time Models with Applications to Term Structure of Interest Rates," Review of Financial Studies, Society for Financial Studies, vol. 18(1), pages 37-84.
    21. James H. Stock & Mark W. Watson, 2007. "Why Has U.S. Inflation Become Harder to Forecast?," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 39(s1), pages 3-33, February.
    22. Maik H. Wolters, 2015. "Evaluating Point and Density Forecasts of DSGE Models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 30(1), pages 74-96, January.
    23. Michael P. Clements, 2004. "Evaluating the Bank of England Density Forecasts of Inflation," Economic Journal, Royal Economic Society, vol. 114(498), pages 844-866, October.
    24. Wolters, Maik H., 2011. "Forecasting under Model Uncertainty," VfS Annual Conference 2011 (Frankfurt, Main): The Order of the World Economy - Lessons from the Crisis 48723, Verein für Socialpolitik / German Economic Association.
    25. Graham Elliott & Allan Timmermann, 2016. "Economic Forecasting," Economics Books, Princeton University Press, edition 1, number 10740.
    26. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
    27. Newey, Whitney & West, Kenneth, 2014. "A simple, positive semi-definite, heteroscedasticity and autocorrelation consistent covariance matrix," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 33(1), pages 125-132.
    28. Inoue, Atsushi, 2001. "Testing For Distributional Change In Time Series," Econometric Theory, Cambridge University Press, vol. 17(1), pages 156-187, February.
    29. González-Rivera, Gloria & Sun, Yingying, 2015. "Generalized autocontours: Evaluation of multivariate density models," International Journal of Forecasting, Elsevier, vol. 31(3), pages 799-814.
    30. Jushan Bai & Serena Ng, 2005. "Tests for Skewness, Kurtosis, and Normality for Time Series Data," Journal of Business & Economic Statistics, American Statistical Association, vol. 23, pages 49-60, January.
    31. Diebold, Francis X & Gunther, Todd A & Tay, Anthony S, 1998. "Evaluating Density Forecasts with Applications to Financial Risk Management," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 863-883, November.
    32. González-Rivera, Gloria & Yoldas, Emre, 2012. "Autocontour-based evaluation of multivariate predictive densities," International Journal of Forecasting, Elsevier, vol. 28(2), pages 328-342.
    33. Amisano, Gianni & Giacomini, Raffaella, 2007. "Comparing Density Forecasts via Weighted Likelihood Ratio Tests," Journal of Business & Economic Statistics, American Statistical Association, vol. 25, pages 177-190, April.
    34. Wolden Bache, Ida & Sofie Jore, Anne & Mitchell, James & Vahey, Shaun P., 2011. "Combining VAR and DSGE forecast densities," Journal of Economic Dynamics and Control, Elsevier, vol. 35(10), pages 1659-1670, October.
    35. N/A, 2016. "The World Economy: Forecast Summary," National Institute Economic Review, National Institute of Economic and Social Research, vol. 238(1), pages 2-2, November.
    36. Hong, Yongmiao & Li, Haitao & Zhao, Feng, 2007. "Can the random walk model be beaten in out-of-sample density forecasts? Evidence from intraday foreign exchange rates," Journal of Econometrics, Elsevier, vol. 141(2), pages 736-776, December.
    37. Diks, Cees & Panchenko, Valentyn & van Dijk, Dick, 2011. "Likelihood-based scoring rules for comparing density forecasts in tails," Journal of Econometrics, Elsevier, vol. 163(2), pages 215-230, August.
    38. Valentina Corradi & Norman Swanson, 2006. "Predictive Density Evaluation. Revised," Departmental Working Papers 200621, Rutgers University, Department of Economics.
    39. Jushan Bai, 2003. "Testing Parametric Conditional Distributions of Dynamic Models," The Review of Economics and Statistics, MIT Press, vol. 85(3), pages 531-549, August.
    40. Gürkaynak, Refet S. & Kisacikoglu, Burçin & Rossi, Barbara, 2013. "Do DSGE Models Forecast More Accurately Out-of-Sample than VAR Models?," CEPR Discussion Papers 9576, C.E.P.R. Discussion Papers.
    41. Rochelle M. Edge & Refet S. Gurkaynak, 2010. "How Useful Are Estimated DSGE Model Forecasts for Central Bankers?," Brookings Papers on Economic Activity, Economic Studies Program, The Brookings Institution, vol. 41(2 (Fall)), pages 209-259.
    42. Rochelle M. Edge & Michael T. Kiley & Jean-Philippe Laforte, 2010. "A comparison of forecast performance between Federal Reserve staff forecasts, simple reduced-form models, and a DSGE model," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(4), pages 720-754.
    43. G. Elliott & C. Granger & A. Timmermann (ed.), 2006. "Handbook of Economic Forecasting," Handbook of Economic Forecasting, Elsevier, edition 1, volume 1, number 1.
    44. Corradi, Valentina & Swanson, Norman R., 2006. "Predictive Density Evaluation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 5, pages 197-284, Elsevier.
    45. repec:hal:journl:peer-00834423 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gergely Ganics & Barbara Rossi & Tatevik Sekhposyan, 2019. "From fixed-event to fixed-horizon density forecasts: obtaining measures of multi-horizon uncertainty from survey density forecasts," Working Papers 1947, Banco de España.
    2. Tobias Adrian & Nina Boyarchenko & Domenico Giannone, 2021. "Multimodality In Macrofinancial Dynamics," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 62(2), pages 861-886, May.
    3. Richard K. Crump & Domenico Giannone & Sean Hundtofte, 2018. "Changing Risk-Return Profiles," Liberty Street Economics 20181004, Federal Reserve Bank of New York.
    4. Matteo Iacopini & Francesco Ravazzolo & Luca Rossini, 2020. "Proper scoring rules for evaluating asymmetry in density forecasting," Papers 2006.11265, arXiv.org, revised Sep 2020.
    5. Plakandaras, Vasilios & Gogas, Periklis & Papadimitriou, Theophilos & Gupta, Rangan, 2019. "A re-evaluation of the term spread as a leading indicator," International Review of Economics & Finance, Elsevier, vol. 64(C), pages 476-492.
    6. Adams, Patrick A. & Adrian, Tobias & Boyarchenko, Nina & Giannone, Domenico, 2021. "Forecasting macroeconomic risks," International Journal of Forecasting, Elsevier, vol. 37(3), pages 1173-1191.
    7. Lhuissier Stéphane, 2022. "Financial Conditions and Macroeconomic Downside Risks in the Euro Area," Working papers 863, Banque de France.
    8. Tobias Adrian & Nina Boyarchenko & Domenico Giannone, 2019. "Vulnerable Growth," American Economic Review, American Economic Association, vol. 109(4), pages 1263-1289, April.
    9. Michael W. McCracken, 2020. "Diverging Tests of Equal Predictive Ability," Econometrica, Econometric Society, vol. 88(4), pages 1753-1754, July.
    10. Taylor, James W., 2020. "A strategic predictive distribution for tests of probabilistic calibration," International Journal of Forecasting, Elsevier, vol. 36(4), pages 1380-1388.
    11. Marcus P. A. Cobb, 2020. "Aggregate density forecasting from disaggregate components using Bayesian VARs," Empirical Economics, Springer, vol. 58(1), pages 287-312, January.
    12. Alex Tagliabracci, 2020. "Asymmetry in the conditional distribution of euro-area inflation," Temi di discussione (Economic working papers) 1270, Bank of Italy, Economic Research and International Relations Area.
    13. Niango Ange Joseph Yapi, 2020. "Exchange rate predictive densities and currency risks: A quantile regression approach," EconomiX Working Papers 2020-16, University of Paris Nanterre, EconomiX.
    14. Korobilis, Dimitris & Landau, Bettina & Musso, Alberto & Phella, Anthoulla, 2021. "The time-varying evolution of inflation risks," Working Paper Series 2600, European Central Bank.
    15. Gary Koop & Stuart McIntyre & James Mitchell & Aubrey Poon, 2022. "Using stochastic hierarchical aggregation constraints to nowcast regional economic aggregates," Working Papers 22-06, Federal Reserve Bank of Cleveland.
    16. Wojciech Charemza & Carlos Diaz Vela & Svetlana Makarova, 2013. "Too many skew normal distributions? The practitioner’s perspective," Discussion Papers in Economics 13/07, Division of Economics, School of Business, University of Leicester.
    17. Jonas Dovern & Hans Manner, 2020. "Order‐invariant tests for proper calibration of multivariate density forecasts," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(4), pages 440-456, June.
    18. Busetti, Fabio & Caivano, Michele & Delle Monache, Davide & Pacella, Claudia, 2021. "The time-varying risk of Italian GDP," Economic Modelling, Elsevier, vol. 101(C).
    19. Barbara Rossi, 2019. "Forecasting in the Presence of Instabilities: How Do We Know Whether Models Predict Well and How to Improve Them," Working Papers 1162, Barcelona Graduate School of Economics.
    20. J. David López-Salido & Francesca Loria, 2020. "Inflation at Risk," Finance and Economics Discussion Series 2020-013, Board of Governors of the Federal Reserve System (U.S.).
    21. Nikoleta Anesti & Ana Beatriz Galvão & Silvia Miranda‐Agrippino, 2022. "Uncertain Kingdom: Nowcasting Gross Domestic Product and its revisions," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(1), pages 42-62, January.
    22. López-Salido, J David & Loria, Francesca, 2019. "Inflation at Risk," CEPR Discussion Papers 14074, C.E.P.R. Discussion Papers.
    23. Anthony Garratt & Ivan Petrella, 2022. "Commodity prices and inflation risk," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(2), pages 392-414, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Rossi, Barbara, 2013. "Advances in Forecasting under Instability," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1203-1324, Elsevier.
    2. Barbara Rossi, 2019. "Forecasting in the presence of instabilities: How do we know whether models predict well and how to improve them," Economics Working Papers 1711, Department of Economics and Business, Universitat Pompeu Fabra, revised Jul 2021.
    3. Rossi, Barbara & Sekhposyan, Tatevik, 2013. "Conditional predictive density evaluation in the presence of instabilities," Journal of Econometrics, Elsevier, vol. 177(2), pages 199-212.
    4. Raffaella Giacomini & Barbara Rossi, 2015. "Forecasting in Nonstationary Environments: What Works and What Doesn't in Reduced-Form and Structural Models," Annual Review of Economics, Annual Reviews, vol. 7(1), pages 207-229, August.
    5. Diks, Cees & Panchenko, Valentyn & van Dijk, Dick, 2011. "Likelihood-based scoring rules for comparing density forecasts in tails," Journal of Econometrics, Elsevier, vol. 163(2), pages 215-230, August.
    6. Jonas Dovern & Hans Manner, 2020. "Order‐invariant tests for proper calibration of multivariate density forecasts," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(4), pages 440-456, June.
    7. Clements, Michael P., 2018. "Are macroeconomic density forecasts informative?," International Journal of Forecasting, Elsevier, vol. 34(2), pages 181-198.
    8. Gaglianone, Wagner Piazza & Marins, Jaqueline Terra Moura, 2017. "Evaluation of exchange rate point and density forecasts: An application to Brazil," International Journal of Forecasting, Elsevier, vol. 33(3), pages 707-728.
    9. Gelain, Paolo & Iskrev, Nikolay & J. Lansing, Kevin & Mendicino, Caterina, 2019. "Inflation dynamics and adaptive expectations in an estimated DSGE model," Journal of Macroeconomics, Elsevier, vol. 59(C), pages 258-277.
    10. Rodríguez, Gabriel, 2017. "Modeling Latin-American stock and Forex markets volatility: Empirical application of a model with random level shifts and genuine long memory," The North American Journal of Economics and Finance, Elsevier, vol. 42(C), pages 393-420.
    11. Anne Sofie Jore & James Mitchell & Shaun P. Vahey, 2010. "Combining forecast densities from VARs with uncertain instabilities," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(4), pages 621-634.
    12. Knut Are Aastveit & Claudia Foroni & Francesco Ravazzolo, 2017. "Density Forecasts With Midas Models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 32(4), pages 783-801, June.
    13. Cees Diks & Valentyn Panchenko & Dick van Dijk, 2008. "Partial Likelihood-Based Scoring Rules for Evaluating Density Forecasts in Tails," Tinbergen Institute Discussion Papers 08-050/4, Tinbergen Institute.
    14. Garratt, Anthony & Mitchell, James & Vahey, Shaun P., 2014. "Measuring output gap nowcast uncertainty," International Journal of Forecasting, Elsevier, vol. 30(2), pages 268-279.
    15. Ida Wolden Bache & James Mitchell & Francesco Ravazzolo & Shaun P. Vahey, 2009. "Macro modelling with many models," Working Paper 2009/15, Norges Bank.
    16. Knut Are Aastveit & James Mitchell & Francesco Ravazzolo & Herman van Dijk, 2018. "The Evolution of Forecast Density Combinations in Economics," Tinbergen Institute Discussion Papers 18-069/III, Tinbergen Institute.
    17. Knut Are Aastveit & Karsten R. Gerdrup & Anne Sofie Jore & Leif Anders Thorsrud, 2014. "Nowcasting GDP in Real Time: A Density Combination Approach," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 32(1), pages 48-68, January.
    18. González-Rivera, Gloria & Yoldas, Emre, 2012. "Autocontour-based evaluation of multivariate predictive densities," International Journal of Forecasting, Elsevier, vol. 28(2), pages 328-342.
    19. Gonçalves Mazzeu, Joao Henrique & González-Rivera, Gloria & Ruiz Ortega, Esther & Veiga, Helena, 2016. "A Bootstrap Approach for Generalized Autocontour Testing," DES - Working Papers. Statistics and Econometrics. WS 23457, Universidad Carlos III de Madrid. Departamento de Estadística.
    20. Rossi, Barbara & Sekhposyan, Tatevik, 2014. "Evaluating predictive densities of US output growth and inflation in a large macroeconomic data set," International Journal of Forecasting, Elsevier, vol. 30(3), pages 662-682.

    More about this item

    Keywords

    Predictive density; Probability integral transform; Kolmogorov–Smirnov test; Cramér–von Mises test; Forecast evaluation;
    All these keywords.

    JEL classification:

    • C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:econom:v:208:y:2019:i:2:p:638-657. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://www.elsevier.com/locate/jeconom .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jeconom .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.