IDEAS home Printed from https://ideas.repec.org/p/cdf/wpaper/2014-11.html
   My bibliography  Save this paper

How good are out of sample forecasting Tests on DSGE models?

Author

Listed:

Abstract

Out-of-sample forecasting tests of DSGE models against time-series benchmarks such as an unrestricted VAR are increasingly used to check a) the specification b) the forecasting capacity of these models. We carry out a Monte Carlo experiment on a widely-used DSGE model to investigate the power of these tests. We find that in specification testing they have weak power relative to an in-sample indirect inference test,this implies that a DSGE model may be badly mis-specified and still improve forecasts from an unrestricted VAR. In testing forecasting capacity they also have quite weak power, particularly on the lefthand tail. By contrast a model that passes an indirect inference test of specification will almost definitely also improve on VAR forecasts.

Suggested Citation

  • Minford, Patrick & Xu, Yongdeng & Zhou, Peng, 2014. "How good are out of sample forecasting Tests on DSGE models?," Cardiff Economics Working Papers E2014/11, Cardiff University, Cardiff Business School, Economics Section.
  • Handle: RePEc:cdf:wpaper:2014/11
    as

    Download full text from publisher

    File URL: http://carbsecon.com/wp/E2014_11.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Kenneth S. Rogoff & Vania Stavrakeva, 2008. "The Continuing Puzzle of Short Horizon Exchange Rate Forecasting," NBER Working Papers 14071, National Bureau of Economic Research, Inc.
    2. Frank Smets & Rafael Wouters, 2007. "Shocks and Frictions in US Business Cycles: A Bayesian DSGE Approach," American Economic Review, American Economic Association, vol. 97(3), pages 586-606, June.
    3. Ince, Onur, 2014. "Forecasting exchange rates out-of-sample with panel methods and real-time data," Journal of International Money and Finance, Elsevier, vol. 43(C), pages 1-18.
    4. Le, Vo Phuong Mai & Meenagh, David & Minford, Patrick, 2012. "What causes banking crises? An empirical investigation," Cardiff Economics Working Papers E2012/14, Cardiff University, Cardiff Business School, Economics Section, revised Apr 2013.
    5. Clark, Todd E. & West, Kenneth D., 2006. "Using out-of-sample mean squared prediction errors to test the martingale difference hypothesis," Journal of Econometrics, Elsevier, vol. 135(1-2), pages 155-186.
    6. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-1084, September.
    7. Le, Vo Phuong Mai & Meenagh, David & Minford, Patrick & Wickens, Michael, 2012. "Testing DSGE models by Indirect inference and other methods: some Monte Carlo experiments," Cardiff Economics Working Papers E2012/15, Cardiff University, Cardiff Business School, Economics Section.
    8. Warne, Anders & Coenen, Günter & Christoffel, Kai, 2010. "Forecasting with DSGE models," Working Paper Series 1185, European Central Bank.
    9. Raffaella Giacomini & Barbara Rossi, 2010. "Forecast comparisons in unstable environments," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(4), pages 595-620.
    10. Michael Wickens, 2014. "How Useful are DSGE Macroeconomic Models for Forecasting?," Open Economies Review, Springer, vol. 25(1), pages 171-193, February.
    11. Michael P. Clements & David F. Hendry, 2005. "Evaluating a Model by Forecast Performance," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 67(s1), pages 931-956, December.
    12. Le, Vo Phuong Mai & Meenagh, David & Minford, Patrick & Wickens, Michael, 2011. "How much nominal rigidity is there in the US economy? Testing a new Keynesian DSGE model using indirect inference," Journal of Economic Dynamics and Control, Elsevier, vol. 35(12), pages 2078-2104.
    13. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    14. Clark, Todd E. & West, Kenneth D., 2007. "Approximately normal tests for equal predictive accuracy in nested models," Journal of Econometrics, Elsevier, vol. 138(1), pages 291-311, May.
    15. Gürkaynak, Refet S. & Kisacikoglu, Burçin & Rossi, Barbara, 2013. "Do DSGE Models Forecast More Accurately Out-of-Sample than VAR Models?," CEPR Discussion Papers 9576, C.E.P.R. Discussion Papers.
    16. Rochelle M. Edge & Refet S. Gurkaynak, 2010. "How Useful Are Estimated DSGE Model Forecasts for Central Bankers?," Brookings Papers on Economic Activity, Economic Studies Program, The Brookings Institution, vol. 41(2 (Fall)), pages 209-259.
    17. Rochelle M. Edge & Michael T. Kiley & Jean-Philippe Laforte, 2010. "A comparison of forecast performance between Federal Reserve staff forecasts, simple reduced-form models, and a DSGE model," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(4), pages 720-754.
    18. Michel Juillard, 2001. "DYNARE: A program for the simulation of rational expectation models," Computing in Economics and Finance 2001 213, Society for Computational Economics.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Patrick Minford & Yi Wang & Peng Zhou, 2020. "Resolving the public-sector wage premium puzzle by indirect inference," Applied Economics, Taylor & Francis Journals, vol. 52(7), pages 726-741, February.
    2. David Meenagh & Patrick Minford & Michael Wickens & Yongdeng Xu, 2019. "Testing DSGE Models by Indirect Inference: a Survey of Recent Findings," Open Economies Review, Springer, vol. 30(3), pages 593-620, July.
    3. Chou, Jenyu & Easaw, Joshy & Minford, Patrick, 2021. "Does Inattentiveness Matter for DSGE Modelling? An Empirical Investigation," Cardiff Economics Working Papers E2021/35, Cardiff University, Cardiff Business School, Economics Section.
    4. Loberto, Michele & Perricone, Chiara, 2017. "Does trend inflation make a difference?," Economic Modelling, Elsevier, vol. 61(C), pages 351-375.
    5. Meenagh, David & Minford, Patrick & Wickens, Michael R., 2021. "Estimating macro models and the potentially misleading nature of Bayesian estimation," CEPR Discussion Papers 15684, C.E.P.R. Discussion Papers.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Rossi, Barbara, 2013. "Advances in Forecasting under Instability," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1203-1324, Elsevier.
    2. Amat, Christophe & Michalski, Tomasz & Stoltz, Gilles, 2018. "Fundamentals and exchange rate forecastability with simple machine learning methods," Journal of International Money and Finance, Elsevier, vol. 88(C), pages 1-24.
    3. Raffaella Giacomini & Barbara Rossi, 2013. "Forecasting in macroeconomics," Chapters, in: Nigar Hashimzade & Michael A. Thornton (ed.), Handbook of Research Methods and Applications in Empirical Macroeconomics, chapter 17, pages 381-408, Edward Elgar Publishing.
    4. Barbara Rossi, 2013. "Exchange Rate Predictability," Journal of Economic Literature, American Economic Association, vol. 51(4), pages 1063-1119, December.
    5. Joseph P. Byrne & Dimitris Korobilis & Pinho J. Ribeiro, 2018. "On The Sources Of Uncertainty In Exchange Rate Predictability," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 59(1), pages 329-357, February.
    6. Barbara Rossi, 2019. "Forecasting in the presence of instabilities: How do we know whether models predict well and how to improve them," Economics Working Papers 1711, Department of Economics and Business, Universitat Pompeu Fabra, revised Jul 2021.
    7. Faust, Jon & Wright, Jonathan H., 2013. "Forecasting Inflation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 2-56, Elsevier.
    8. Byrne, Joseph P. & Korobilis, Dimitris & Ribeiro, Pinho J., 2016. "Exchange rate predictability in a changing world," Journal of International Money and Finance, Elsevier, vol. 62(C), pages 1-24.
    9. Gürkaynak, Refet S. & Kisacikoglu, Burçin & Rossi, Barbara, 2013. "Do DSGE Models Forecast More Accurately Out-of-Sample than VAR Models?," CEPR Discussion Papers 9576, C.E.P.R. Discussion Papers.
    10. Ciner, Cetin, 2017. "Predicting white metal prices by a commodity sensitive exchange rate," International Review of Financial Analysis, Elsevier, vol. 52(C), pages 309-315.
    11. Byrne, Joseph P. & Korobilis, Dimitris & Ribeiro, Pinho J., 2014. "On the Sources of Uncertainty in Exchange Rate Predictability," 2007 Annual Meeting, July 29-August 1, 2007, Portland, Oregon TN 2015-24, American Agricultural Economics Association (New Name 2008: Agricultural and Applied Economics Association).
    12. Ayse Kabukcuoglu & Enrique Martínez-García, 2016. "What Helps Forecast U.S. Inflation?—Mind the Gap!," Koç University-TUSIAD Economic Research Forum Working Papers 1615, Koc University-TUSIAD Economic Research Forum.
    13. Liu, Chunping & Minford, Patrick, 2014. "Comparing behavioural and rational expectations for the US post-war economy," Economic Modelling, Elsevier, vol. 43(C), pages 407-415.
    14. Tanya Molodtsova & Alex Nikolsko-Rzhevskyy & David H. Papell, 2011. "Taylor Rules and the Euro," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 43, pages 535-552, March.
    15. Wang, Rudan & Morley, Bruce & Stamatogiannis, Michalis P., 2019. "Forecasting the exchange rate using nonlinear Taylor rule based models," International Journal of Forecasting, Elsevier, vol. 35(2), pages 429-442.
    16. Rossi, Barbara & Sekhposyan, Tatevik, 2011. "Understanding models' forecasting performance," Journal of Econometrics, Elsevier, vol. 164(1), pages 158-172, September.
    17. Kenneth S. Rogoff & Vania Stavrakeva, 2008. "The Continuing Puzzle of Short Horizon Exchange Rate Forecasting," NBER Working Papers 14071, National Bureau of Economic Research, Inc.
    18. Joseph Agyapong, 2021. "Application of Taylor Rule Fundamentals in Forecasting Exchange Rates," Economies, MDPI, vol. 9(2), pages 1-27, June.
    19. Wieland, Volker & Wolters, Maik, 2013. "Forecasting and Policy Making," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 239-325, Elsevier.
    20. Ince, Onur & Molodtsova, Tanya & Papell, David H., 2016. "Taylor rule deviations and out-of-sample exchange rate predictability," Journal of International Money and Finance, Elsevier, vol. 69(C), pages 22-44.

    More about this item

    Keywords

    Out of sample forecasts; DSGE; VAR; specification tests; indirect inference; forecast performance;
    All these keywords.

    JEL classification:

    • E10 - Macroeconomics and Monetary Economics - - General Aggregative Models - - - General
    • E17 - Macroeconomics and Monetary Economics - - General Aggregative Models - - - Forecasting and Simulation: Models and Applications

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cdf:wpaper:2014/11. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/ecscfuk.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Yongdeng Xu (email available below). General contact details of provider: https://edirc.repec.org/data/ecscfuk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.