IDEAS home Printed from https://ideas.repec.org/p/bge/wpaper/784.html
   My bibliography  Save this paper

Model Comparisons in Unstable Environments

Author

Listed:
  • Raffaella Giacomini
  • Barbara Rossi

Abstract

The goal of this paper is to develop formal tests to evaluate the relative in-sample performance of two competing, misspecified, non-nested models in the presence of possible data instability. Compared to previous approaches to model selection, which are based on measures of global performance, we focus on the local relative performance of the models. We propose three tests that are based on different measures of local performance and that correspond to different null and alternative hypotheses. The empirical application provides insights into the time variation in the performance of a representative DSGE model of the European economy relative to that of VARs.

Suggested Citation

  • Raffaella Giacomini & Barbara Rossi, 2014. "Model Comparisons in Unstable Environments," Working Papers 784, Barcelona Graduate School of Economics.
  • Handle: RePEc:bge:wpaper:784
    as

    Download full text from publisher

    File URL: http://www.barcelonagse.eu/sites/default/files/working_paper_pdfs/784_0.pdf
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. Wei Biao Wu & Zhibiao Zhao, 2007. "Inference of trends in time series," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 69(3), pages 391-410.
    2. Ulrich K. Müller, 2011. "Efficient Tests Under a Weak Convergence Assumption," Econometrica, Econometric Society, vol. 79(2), pages 395-435, March.
    3. Rossi, Barbara, 2005. "Optimal Tests For Nested Model Selection With Underlying Parameter Instability," Econometric Theory, Cambridge University Press, vol. 21(05), pages 962-990, October.
    4. Hansen, Bruce E., 2000. "Testing for structural change in conditional models," Journal of Econometrics, Elsevier, pages 93-115.
    5. Andrews, Donald W K & Ploberger, Werner, 1994. "Optimal Tests When a Nuisance Parameter Is Present Only under the Alternative," Econometrica, Econometric Society, pages 1383-1414.
    6. Raffaella Giacomini & Barbara Rossi, 2010. "Forecast comparisons in unstable environments," Journal of Applied Econometrics, John Wiley & Sons, Ltd., pages 595-620.
    7. Vuong, Quang H, 1989. "Likelihood Ratio Tests for Model Selection and Non-nested Hypotheses," Econometrica, Econometric Society, vol. 57(2), pages 307-333, March.
    8. Andrews, Donald W K, 1993. "Tests for Parameter Instability and Structural Change with Unknown Change Point," Econometrica, Econometric Society, vol. 61(4), pages 821-856, July.
    9. Zhongjun Qu & Pierre Perron, 2007. "Estimating and Testing Structural Changes in Multivariate Regressions," Econometrica, Econometric Society, vol. 75(2), pages 459-502, March.
    10. Wooldridge, Jeffrey M. & White, Halbert, 1988. "Some Invariance Principles and Central Limit Theorems for Dependent Heterogeneous Processes," Econometric Theory, Cambridge University Press, vol. 4(02), pages 210-230, August.
    11. Cavaliere, Giuseppe & Taylor, A.M. Robert, 2005. "Stationarity Tests Under Time-Varying Second Moments," Econometric Theory, Cambridge University Press, vol. 21(06), pages 1112-1129, December.
    12. Douglas Rivers & Quang Vuong, 2002. "Model selection tests for nonlinear dynamic models," Econometrics Journal, Royal Economic Society, vol. 5(1), pages 1-39, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Rossi, Barbara, 2013. "Advances in Forecasting under Instability," Handbook of Economic Forecasting, Elsevier.
    2. Chitolina, Lia & Foguel, Miguel Nathan & Menezes-Filho, Naercio Aquino, 2016. "The impact of the expansion of the bolsa familia program on the time allocation of youths and their parents," Revista Brasileira de Economia - RBE, FGV/EPGE - Escola Brasileira de Economia e Finanças, Getulio Vargas Foundation (Brazil).
    3. Yin, Anwen, 2015. "Forecasting and model averaging with structural breaks," ISU General Staff Papers 201501010800005727, Iowa State University, Department of Economics.
    4. repec:eee:dyncon:v:82:y:2017:i:c:p:67-82 is not listed on IDEAS
    5. Rossi, José Luiz Júnior, 2014. "The Usefulness of Financial Variables in Predicting Exchange Rate Movements," Insper Working Papers wpe_332, Insper Working Paper, Insper Instituto de Ensino e Pesquisa.
    6. Giacomini, Raffaella, 2014. "Economic theory and forecasting: lessons from the literature," CEPR Discussion Papers 10201, C.E.P.R. Discussion Papers.
    7. Barbara Rossi & Atsushi Inoue, 2012. "Out-of-Sample Forecast Tests Robust to the Choice of Window Size," Journal of Business & Economic Statistics, Taylor & Francis Journals, pages 432-453.
    8. Chollete, Loran & Schmeidler, David, 2014. "Extreme Events and the Origin of Central Bank Priors," UiS Working Papers in Economics and Finance 2014/15, University of Stavanger.
    9. Ana Beatriz Galvão & Liudas Giraitis & George Kapetanios & Katerina Petrova, 2015. "A Bayesian Local Likelihood Method for Modelling Parameter Time Variation in DSGE Models," Working Papers 770, Queen Mary University of London, School of Economics and Finance.
    10. Leandro M. Magnusson & Sophocles Mavroeidis, 2014. "Identification Using Stability Restrictions," Econometrica, Econometric Society, vol. 82, pages 1799-1851, September.
    11. Gürkaynak, Refet S. & Kisacikoglu, Burçin & Rossi, Barbara, 2013. "Do DSGE Models Forecast More Accurately Out-of-Sample than VAR Models?," CEPR Discussion Papers 9576, C.E.P.R. Discussion Papers.
    12. Raffaella Giacomini, 2014. "Economic theory and forecasting: lessons from the literature," CeMMAP working papers CWP41/14, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    13. Raffaella Giacomini & Barbara Rossi, 2010. "Forecast comparisons in unstable environments," Journal of Applied Econometrics, John Wiley & Sons, Ltd., pages 595-620.
    14. Weber, Enzo & Zika, Gerd, 2013. "Labour market forecasting : is disaggregation useful?," IAB Discussion Paper 201314, Institut für Arbeitsmarkt- und Berufsforschung (IAB), Nürnberg [Institute for Employment Research, Nuremberg, Germany].
    15. Cantore, Cristiano & Ferroni, Filippo & León-Ledesma, Miguel A., 2017. "The dynamics of hours worked and technology," Journal of Economic Dynamics and Control, Elsevier, vol. 82(C), pages 67-82.
    16. Roberta Cardani & Alessia Paccagnini & Stefania Villa, 2015. "Forecasting with Instabilities: an Application to DSGE Models with Financial Frictions," Working Papers 201523, School of Economics, University College Dublin.

    More about this item

    Keywords

    model selection tests; misspecification; structural change; Kullback-Leibler information criterion;

    JEL classification:

    • C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bge:wpaper:784. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Bruno Guallar). General contact details of provider: http://edirc.repec.org/data/bargses.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.