IDEAS home Printed from https://ideas.repec.org/a/eee/intfor/v33y2017i2p416-432.html
   My bibliography  Save this article

Density forecast evaluation in unstable environments

Author

Listed:
  • González-Rivera, Gloria
  • Sun, Yingying

Abstract

We propose a density forecast evaluation method in the presence of instabilities, which are defined as breaks in any conditional moment of interest and/or in the functional form of the conditional density of the process. Within the framework of the autocontour-based tests proposed by González-Rivera et al. (2011) and González-Rivera and Sun (2015), we construct Sup- and Ave-type tests, calculated over a collection of subsamples in the evaluation period. These tests have asymptotic distributions that are nuisance-parameter free and they are correctly sized and very powerful for detecting breaks in the parameters of the conditional mean and conditional variance. A power comparison with the tests of Rossi and Sekhposyan (2013) shows that our tests are more powerful across the models considered in their work. We analyze the stability of a dynamic Phillips curve and find that the best one-step-ahead density forecast of changes in inflation is generated by a Markov switching model that allows state shifts in the mean and variance of inflation changes as well as in the coefficient that links inflation and unemployment.

Suggested Citation

  • González-Rivera, Gloria & Sun, Yingying, 2017. "Density forecast evaluation in unstable environments," International Journal of Forecasting, Elsevier, vol. 33(2), pages 416-432.
    • Handle: RePEc:eee:intfor:v:33:y:2017:i:2:p:416-432
    DOI: 10.1016/j.ijforecast.2016.10.003
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0169207016301200
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ijforecast.2016.10.003?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Elliott, Graham & Muller, Ulrich K., 2007. "Confidence sets for the date of a single break in linear time series regressions," Journal of Econometrics, Elsevier, vol. 141(2), pages 1196-1218, December.
    2. Stock, James H. & Watson, Mark W., 1999. "Forecasting inflation," Journal of Monetary Economics, Elsevier, vol. 44(2), pages 293-335, October.
    3. Pesaran, M. Hashem & Timmermann, Allan, 2002. "Market timing and return prediction under model instability," Journal of Empirical Finance, Elsevier, vol. 9(5), pages 495-510, December.
    4. González-Rivera, Gloria & Sun, Yingying, 2015. "Generalized autocontours: Evaluation of multivariate density models," International Journal of Forecasting, Elsevier, vol. 31(3), pages 799-814.
    5. Donald W. K. Andrews, 2003. "Tests for Parameter Instability and Structural Change with Unknown Change Point: A Corrigendum," Econometrica, Econometric Society, vol. 71(1), pages 395-397, January.
    6. Valentina Corradi & Norman Swanson, 2006. "Predictive Density Evaluation. Revised," Departmental Working Papers 200621, Rutgers University, Department of Economics.
    7. Amisano, Gianni & Giacomini, Raffaella, 2007. "Comparing Density Forecasts via Weighted Likelihood Ratio Tests," Journal of Business & Economic Statistics, American Statistical Association, vol. 25, pages 177-190, April.
    8. Rossi, Barbara & Sekhposyan, Tatevik, 2013. "Conditional predictive density evaluation in the presence of instabilities," Journal of Econometrics, Elsevier, vol. 177(2), pages 199-212.
    9. Amit Goyal & Ivo Welch, 2003. "Predicting the Equity Premium with Dividend Ratios," Management Science, INFORMS, vol. 49(5), pages 639-654, May.
    10. Kenneth S. Rogoff & Vania Stavrakeva, 2008. "The Continuing Puzzle of Short Horizon Exchange Rate Forecasting," NBER Working Papers 14071, National Bureau of Economic Research, Inc.
    11. Barbara Rossi & Tatevik Sekhposyan, 2011. "Forecast Optimality Tests in the Presence of Instabilities," Working Papers 11-18, Duke University, Department of Economics.
    12. Andrews, Donald W K & Ploberger, Werner, 1994. "Optimal Tests When a Nuisance Parameter Is Present Only under the Alternative," Econometrica, Econometric Society, vol. 62(6), pages 1383-1414, November.
    13. Gloria González-Rivera & Zeynep Senyuz & Emre Yoldas, 2011. "Autocontours: Dynamic Specification Testing," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 29(1), pages 186-200, January.
    14. M. Hashem Pesaran & Davide Pettenuzzo & Allan Timmermann, 2006. "Forecasting Time Series Subject to Multiple Structural Breaks," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 73(4), pages 1057-1084.
    15. Rossi, Barbara, 2006. "Are Exchange Rates Really Random Walks? Some Evidence Robust To Parameter Instability," Macroeconomic Dynamics, Cambridge University Press, vol. 10(1), pages 20-38, February.
    16. Diebold, Francis X & Gunther, Todd A & Tay, Anthony S, 1998. "Evaluating Density Forecasts with Applications to Financial Risk Management," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 863-883, November.
    17. González-Rivera, Gloria & Yoldas, Emre, 2012. "Autocontour-based evaluation of multivariate predictive densities," International Journal of Forecasting, Elsevier, vol. 28(2), pages 328-342.
    18. Tatevik Sekhposyan & Barbara Rossi, 2008. "Has modelsí forecasting performance for US output growth and inflation changed over time, and when?," Working Papers 09-02, Duke University, Department of Economics.
    19. Andrews, Donald W K, 1993. "Tests for Parameter Instability and Structural Change with Unknown Change Point," Econometrica, Econometric Society, vol. 61(4), pages 821-856, July.
    20. James H. Stock & Mark W.Watson, 2003. "Forecasting Output and Inflation: The Role of Asset Prices," Journal of Economic Literature, American Economic Association, vol. 41(3), pages 788-829, September.
    21. Corradi, Valentina & Swanson, Norman R., 2006. "Predictive Density Evaluation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 5, pages 197-284, Elsevier.
    22. Paye, Bradley S. & Timmermann, Allan, 2006. "Instability of return prediction models," Journal of Empirical Finance, Elsevier, vol. 13(3), pages 274-315, June.
    23. Rossi, Barbara & Sekhposyan, Tatevik, 2010. "Have economic models' forecasting performance for US output growth and inflation changed over time, and when?," International Journal of Forecasting, Elsevier, vol. 26(4), pages 808-835, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Emilio Zanetti Chini, 2018. "Forecaster’s utility and forecasts coherence," DEM Working Papers Series 145, University of Pavia, Department of Economics and Management.
    2. Marcus P. A. Cobb, 2020. "Aggregate density forecasting from disaggregate components using Bayesian VARs," Empirical Economics, Springer, vol. 58(1), pages 287-312, January.
    3. Szafranek, Karol, 2019. "Bagged neural networks for forecasting Polish (low) inflation," International Journal of Forecasting, Elsevier, vol. 35(3), pages 1042-1059.
    4. Khowaja Kainat & Saef Danial & Sizov Sergej & Härdle Wolfgang Karl, 2024. "Scenario based merger & acquisition forecasting," Management & Marketing, Sciendo, vol. 19(4), pages 579-600.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Rossi, Barbara, 2013. "Advances in Forecasting under Instability," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1203-1324, Elsevier.
    2. Rossi, Barbara & Sekhposyan, Tatevik, 2014. "Evaluating predictive densities of US output growth and inflation in a large macroeconomic data set," International Journal of Forecasting, Elsevier, vol. 30(3), pages 662-682.
    3. Raffaella Giacomini & Barbara Rossi, 2015. "Forecasting in Nonstationary Environments: What Works and What Doesn't in Reduced-Form and Structural Models," Annual Review of Economics, Annual Reviews, vol. 7(1), pages 207-229, August.
    4. Raffaella Giacomini & Barbara Rossi, 2013. "Forecasting in macroeconomics," Chapters, in: Nigar Hashimzade & Michael A. Thornton (ed.), Handbook of Research Methods and Applications in Empirical Macroeconomics, chapter 17, pages 381-408, Edward Elgar Publishing.
    5. Rossi, Barbara & Sekhposyan, Tatevik, 2019. "Alternative tests for correct specification of conditional predictive densities," Journal of Econometrics, Elsevier, vol. 208(2), pages 638-657.
    6. Barbara Rossi, 2021. "Forecasting in the Presence of Instabilities: How We Know Whether Models Predict Well and How to Improve Them," Journal of Economic Literature, American Economic Association, vol. 59(4), pages 1135-1190, December.
    7. Boot, Tom & Pick, Andreas, 2020. "Does modeling a structural break improve forecast accuracy?," Journal of Econometrics, Elsevier, vol. 215(1), pages 35-59.
    8. Gonçalves Mazzeu, Joao Henrique & González-Rivera, Gloria & Ruiz Ortega, Esther & Veiga, Helena, 2016. "A Bootstrap Approach for Generalized Autocontour Testing," DES - Working Papers. Statistics and Econometrics. WS 23457, Universidad Carlos III de Madrid. Departamento de Estadística.
    9. Cai, Lili & Swanson, Norman R., 2011. "In- and out-of-sample specification analysis of spot rate models: Further evidence for the period 1982-2008," Journal of Empirical Finance, Elsevier, vol. 18(4), pages 743-764, September.
    10. Rossi, Barbara & Sekhposyan, Tatevik, 2013. "Conditional predictive density evaluation in the presence of instabilities," Journal of Econometrics, Elsevier, vol. 177(2), pages 199-212.
    11. Marcus P. A. Cobb, 2020. "Aggregate density forecasting from disaggregate components using Bayesian VARs," Empirical Economics, Springer, vol. 58(1), pages 287-312, January.
    12. Narayan, Seema & Smyth, Russell, 2015. "The financial econometrics of price discovery and predictability," International Review of Financial Analysis, Elsevier, vol. 42(C), pages 380-393.
    13. Jiawen Xu & Pierre Perron, 2015. "Forecasting in the presence of in and out of sample breaks," Boston University - Department of Economics - Working Papers Series wp2015-012, Boston University - Department of Economics.
    14. Inoue, Atsushi & Jin, Lu & Rossi, Barbara, 2017. "Rolling window selection for out-of-sample forecasting with time-varying parameters," Journal of Econometrics, Elsevier, vol. 196(1), pages 55-67.
    15. João Henrique G. Mazzeu & Gloria González-Rivera & Esther Ruiz & Helena Veiga, 2020. "A bootstrap approach for generalized Autocontour testing Implications for VIX forecast densities," Econometric Reviews, Taylor & Francis Journals, vol. 39(10), pages 971-990, November.
    16. Tom Boot & Andreas Pick, 2017. "A near optimal test for structural breaks when forecasting under square error loss," Tinbergen Institute Discussion Papers 17-039/III, Tinbergen Institute.
    17. Sjoerd van den Hauwe & Richard Paap & Dick J.C. van Dijk, 2011. "An Alternative Bayesian Approach to Structural Breaks in Time Series Models," Tinbergen Institute Discussion Papers 11-023/4, Tinbergen Institute.
    18. Hartmann, Daniel & Kempa, Bernd & Pierdzioch, Christian, 2008. "Economic and financial crises and the predictability of U.S. stock returns," Journal of Empirical Finance, Elsevier, vol. 15(3), pages 468-480, June.
    19. Andersen, Torben G. & Varneskov, Rasmus T., 2022. "Testing for parameter instability and structural change in persistent predictive regressions," Journal of Econometrics, Elsevier, vol. 231(2), pages 361-386.
    20. Alessandro Casini & Pierre Perron, 2018. "Structural Breaks in Time Series," Boston University - Department of Economics - Working Papers Series WP2019-02, Boston University - Department of Economics.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:intfor:v:33:y:2017:i:2:p:416-432. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/ijforecast .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.