IDEAS home Printed from https://ideas.repec.org/a/eee/ecmode/v79y2019icp242-246.html
   My bibliography  Save this article

Rationality tests in the presence of instabilities in finite samples

Author

Listed:
  • El-Shagi, Makram

Abstract

The importance of expectations in modern macroeconomic models and in particular of policy makers expectations for forward looking policy rules has generated a lot of interest in time series of professional forecasts (including central bank staff forecasts). This has spawned a large literature on the evaluation of forecasts that are not model based or where the model is unknown. Although, the available time series of historical forecasts are typically short, this literature has so far mostly disregarded the small sample properties of the proposed tests and estimators. In this paper we fill this gap in the literature, focusing on a set of recently proposed rationality tests for unstable environments. Using a Monte Carlo study we demonstrate that the asymptotic tests are substantially oversized in finite samples including any sample size that is practically available. We provide finite sample adjusted critical values, that allow those tests to be properly applied to sample sizes of typically available forecasts such as the Survey of Professional Forecasters, the Federal Open Market Committee. The critical values we provide will help to avoid false rejections using those data.

Suggested Citation

  • El-Shagi, Makram, 2019. "Rationality tests in the presence of instabilities in finite samples," Economic Modelling, Elsevier, vol. 79(C), pages 242-246.
  • Handle: RePEc:eee:ecmode:v:79:y:2019:i:c:p:242-246
    DOI: 10.1016/j.econmod.2018.11.011
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0264999318309891
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.econmod.2018.11.011?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Raffaella Giacomini & Halbert White, 2006. "Tests of Conditional Predictive Ability," Econometrica, Econometric Society, vol. 74(6), pages 1545-1578, November.
    2. Andrade, Philippe & Le Bihan, Hervé, 2013. "Inattentive professional forecasters," Journal of Monetary Economics, Elsevier, vol. 60(8), pages 967-982.
    3. Benchimol, Jonathan & El-Shagi, Makram, 2020. "Forecast performance in times terrorism," Economic Modelling, Elsevier, vol. 91(C), pages 386-402.
    4. Barbara Rossi & Tatevik Sekhposyan, 2016. "Forecast Rationality Tests in the Presence of Instabilities, with Applications to Federal Reserve and Survey Forecasts," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 31(3), pages 507-532, April.
    5. Richard Clarida & Jordi Galí & Mark Gertler, 2000. "Monetary Policy Rules and Macroeconomic Stability: Evidence and Some Theory," The Quarterly Journal of Economics, Oxford University Press, vol. 115(1), pages 147-180.
    6. Antonello D'Agostino & Karl Whelan, 2008. "Federal Reserve Information During the Great Moderation," Journal of the European Economic Association, MIT Press, vol. 6(2-3), pages 609-620, 04-05.
    7. Lahiri, Kajal & Monokroussos, George & Zhao, Yongchen, 2013. "The yield spread puzzle and the information content of SPF forecasts," Economics Letters, Elsevier, vol. 118(1), pages 219-221.
    8. Raffaella Giacomini & Barbara Rossi, 2010. "Forecast comparisons in unstable environments," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(4), pages 595-620.
    9. El-Shagi, Makram & Giesen, Sebastian & Jung, Alexander, 2016. "Revisiting the relative forecast performances of Fed staff and private forecasters: A dynamic approach," International Journal of Forecasting, Elsevier, vol. 32(2), pages 313-323.
    10. Lutz Kilian, 1998. "Small-Sample Confidence Intervals For Impulse Response Functions," The Review of Economics and Statistics, MIT Press, vol. 80(2), pages 218-230, May.
    11. Makram El-Shagi & Sebastian Giesen, 2013. "Testing for Structural Breaks at Unknown Time: A Steeplechase," Computational Economics, Springer;Society for Computational Economics, vol. 41(1), pages 101-123, January.
    12. Christopher D. Carroll, 2003. "Macroeconomic Expectations of Households and Professional Forecasters," The Quarterly Journal of Economics, Oxford University Press, vol. 118(1), pages 269-298.
    13. Roberts, John M, 1995. "New Keynesian Economics and the Phillips Curve," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 27(4), pages 975-984, November.
    14. Michael D. Bauer & Glenn D. Rudebusch & Jing Cynthia Wu, 2012. "Correcting Estimation Bias in Dynamic Term Structure Models," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 30(3), pages 454-467, April.
    15. El-Shagi, Makram, 2017. "Dealing with small sample bias in post-crisis samples," Economic Modelling, Elsevier, vol. 65(C), pages 1-8.
    16. Faust, Jon & Wright, Jonathan H., 2009. "Comparing Greenbook and Reduced Form Forecasts Using a Large Realtime Dataset," Journal of Business & Economic Statistics, American Statistical Association, vol. 27(4), pages 468-479.
    17. West, Kenneth D & McCracken, Michael W, 1998. "Regression-Based Tests of Predictive Ability," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 817-840, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Benchimol, Jonathan & El-Shagi, Makram, 2020. "Forecast performance in times terrorism," Economic Modelling, Elsevier, vol. 91(C), pages 386-402.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Benchimol, Jonathan & El-Shagi, Makram, 2020. "Forecast performance in times terrorism," Economic Modelling, Elsevier, vol. 91(C), pages 386-402.
    2. Jonathan Benchimol & Makram El-Shagi & Yossi Saadon, 2020. "Do Expert Experience and Characteristics Affect Inflation Forecasts?," CFDS Discussion Paper Series 2020/6, Center for Financial Development and Stability at Henan University, Kaifeng, Henan, China.
    3. Rossi, Barbara, 2013. "Advances in Forecasting under Instability," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1203-1324, Elsevier.
    4. Andrade, Philippe & Crump, Richard K. & Eusepi, Stefano & Moench, Emanuel, 2016. "Fundamental disagreement," Journal of Monetary Economics, Elsevier, vol. 83(C), pages 106-128.
    5. Martínez-Martin, Jaime & Morris, Richard & Onorante, Luca & Piersanti, Fabio M., 2019. "Merging structural and reduced-form models for forecasting: opening the DSGE-VAR box," Working Paper Series 2335, European Central Bank.
    6. Ricardo Nunes, 2010. "Inflation Dynamics: The Role of Expectations," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 42(6), pages 1161-1172, September.
    7. Beqiraj Elton & Di Bartolomeo Giovanni & Serpieri Carolina, 2017. "Bounded-rationality and heterogeneous agents: Long or short forecasters?," wp.comunite 00132, Department of Communication, University of Teramo.
    8. Nunes, Ricardo, 2009. "Learning The Inflation Target," Macroeconomic Dynamics, Cambridge University Press, vol. 13(2), pages 167-188, April.
    9. Rossi, Barbara & Sekhposyan, Tatevik, 2011. "Understanding models' forecasting performance," Journal of Econometrics, Elsevier, vol. 164(1), pages 158-172, September.
    10. Kontogeorgos, Georgios & Lambrias, Kyriacos, 2019. "An analysis of the Eurosystem/ECB projections," Working Paper Series 2291, European Central Bank.
    11. Costas Milas & Ruthira Naraidoo, 2009. "Financial Market Conditions, Real Time, Nonlinearity and European Central Bank Monetary Policy: In-Sample and Out-of-Sample Assessment," Working Paper series 42_09, Rimini Centre for Economic Analysis.
    12. Francesco Ravazzolo & Philip Rothman, 2013. "Oil and U.S. GDP: A Real-Time Out-of-Sample Examination," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 45(2-3), pages 449-463, March.
    13. Kim, Insu & Kim, Young Se, 2019. "Inattentive agents and inflation forecast error dynamics: A Bayesian DSGE approach," Journal of Macroeconomics, Elsevier, vol. 62(C).
    14. Raffaella Giacomini & Barbara Rossi, 2015. "Forecasting in Nonstationary Environments: What Works and What Doesn't in Reduced-Form and Structural Models," Annual Review of Economics, Annual Reviews, vol. 7(1), pages 207-229, August.
    15. Beqiraj, Elton & Di Bartolomeo, Giovanni & Di Pietro, Marco, 2019. "Beliefs formation and the puzzle of forward guidance power," Journal of Macroeconomics, Elsevier, vol. 60(C), pages 20-32.
    16. Castle, Jennifer L. & Clements, Michael P. & Hendry, David F., 2013. "Forecasting by factors, by variables, by both or neither?," Journal of Econometrics, Elsevier, vol. 177(2), pages 305-319.
    17. Barbara Rossi, 2019. "Forecasting in the presence of instabilities: How do we know whether models predict well and how to improve them," Economics Working Papers 1711, Department of Economics and Business, Universitat Pompeu Fabra.
    18. Berge, Travis J. & Chang, Andrew C. & Sinha, Nitish R., 2019. "Evaluating the conditionality of judgmental forecasts," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1627-1635.
    19. Marc Giannoni & Michael Woodford, 2004. "Optimal Inflation-Targeting Rules," NBER Chapters, in: The Inflation-Targeting Debate, pages 93-172, National Bureau of Economic Research, Inc.
    20. Travis J. Berge & Andrew C. Chang & Nitish R. Sinha, 2019. "Evaluating the Conditionality of Judgmental Forecasts," Finance and Economics Discussion Series 2019-002, Board of Governors of the Federal Reserve System (U.S.).

    More about this item

    Keywords

    Macroeconomic forecasting; Unstable environment; Finite sample; Evaluating forecasts; Survey forecasts;
    All these keywords.

    JEL classification:

    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ecmode:v:79:y:2019:i:c:p:242-246. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://www.elsevier.com/locate/inca/30411 .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/inca/30411 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.