IDEAS home Printed from https://ideas.repec.org/p/fip/fedkrw/rwp01-14.html
   My bibliography  Save this paper

Evaluating long-horizon forecasts

Author

Listed:
  • Todd E. Clark
  • Michael W. McCracken

Abstract

This paper examines the asymptotic and finite-sample properties of tests of equal forecast accuracy and encompassing applied to predictions from nested long-horizon regression models. We first derive the asymptotic distributions of a set of tests of equal forecast accuracy and encompassing, showing that the tests have non-standard distributions that depend on the parameters of the data-generating process. Using a simple parametric bootstrap for inference, we then conduct Monte Carlo simulations of a range of data-generating processes to examine the finite-sample size and power of the tests. In these simulations, the bootstrap yields tests with good finite-sample size and power properties, with the encompassing test proposed by Clark and McCracken (2001a) having superior power. The paper concludes with a reexamination of the predictive content of capacity utilization for core inflation.

Suggested Citation

  • Todd E. Clark & Michael W. McCracken, 2001. "Evaluating long-horizon forecasts," Research Working Paper RWP 01-14, Federal Reserve Bank of Kansas City.
  • Handle: RePEc:fip:fedkrw:rwp01-14
    as

    Download full text from publisher

    File URL: http://www.kansascityfed.org/Publicat/Reswkpap/PDF/rwp01-14.pdf
    Download Restriction: no

    References listed on IDEAS

    as
    1. Chao, John & Corradi, Valentina & Swanson, Norman R., 2001. "Out-Of-Sample Tests For Granger Causality," Macroeconomic Dynamics, Cambridge University Press, vol. 5(04), pages 598-620, September.
    2. Orphanides, Athanasios & van Norden, Simon, 2005. "The Reliability of Inflation Forecasts Based on Output Gap Estimates in Real Time," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 37(3), pages 583-601, June.
    3. Clarida, Richard H. & Sarno, Lucio & Taylor, Mark P. & Valente, Giorgio, 2003. "The out-of-sample success of term structure models as exchange rate predictors: a step beyond," Journal of International Economics, Elsevier, vol. 60(1), pages 61-83, May.
    4. Kilian, Lutz, 1999. "Exchange Rates and Monetary Fundamentals: What Do We Learn from Long-Horizon Regressions?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 14(5), pages 491-510, Sept.-Oct.
    5. Shintani, Mototsugu, 2005. "Nonlinear Forecasting Analysis Using Diffusion Indexes: An Application to Japan," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 37(3), pages 517-538, June.
    6. Kilian, Lutz & Taylor, Mark P., 2003. "Why is it so difficult to beat the random walk forecast of exchange rates?," Journal of International Economics, Elsevier, vol. 60(1), pages 85-107, May.
    7. Rossi, Barbara, 2005. "Optimal Tests For Nested Model Selection With Underlying Parameter Instability," Econometric Theory, Cambridge University Press, vol. 21(05), pages 962-990, October.
    8. Cheung, Yin-Wong & Chinn, Menzie D. & Pascual, Antonio Garcia, 2005. "Empirical exchange rate models of the nineties: Are any fit to survive?," Journal of International Money and Finance, Elsevier, vol. 24(7), pages 1150-1175, November.
    9. repec:cup:macdyn:v:5:y:2001:i:4:p:598-620 is not listed on IDEAS
    10. Jeremy Berkowitz & Lutz Kilian, 2000. "Recent developments in bootstrapping time series," Econometric Reviews, Taylor & Francis Journals, vol. 19(1), pages 1-48.
    11. Todd E. Clark, 2004. "Can out-of-sample forecast comparisons help prevent overfitting?," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 23(2), pages 115-139.
    12. Clark, Todd E. & McCracken, Michael W., 2001. "Tests of equal forecast accuracy and encompassing for nested models," Journal of Econometrics, Elsevier, vol. 105(1), pages 85-110, November.
    13. Hodrick, Robert J, 1992. "Dividend Yields and Expected Stock Returns: Alternative Procedures for Inference and Measurement," Review of Financial Studies, Society for Financial Studies, vol. 5(3), pages 357-386.
    14. Chevillon, Guillaume & Hendry, David F., 2005. "Non-parametric direct multi-step estimation for forecasting economic processes," International Journal of Forecasting, Elsevier, vol. 21(2), pages 201-218.
    15. Atsushi Inoue & Lutz Kilian, 2005. "In-Sample or Out-of-Sample Tests of Predictability: Which One Should We Use?," Econometric Reviews, Taylor & Francis Journals, vol. 23(4), pages 371-402.
    16. Clive Granger & Yongil Jeon, 2004. "Forecasting Performance of Information Criteria with Many Macro Series," Journal of Applied Statistics, Taylor & Francis Journals, vol. 31(10), pages 1227-1240.
    17. West, Kenneth D, 2001. "Tests for Forecast Encompassing When Forecasts Depend on Estimated Regression Parameters," Journal of Business & Economic Statistics, American Statistical Association, vol. 19(1), pages 29-33, January.
    18. Jan J. J. Groen, 1999. "Long horizon predictability of exchange rates: Is it for real?," Empirical Economics, Springer, vol. 24(3), pages 451-469.
    19. Goncalves, Silvia & Kilian, Lutz, 2004. "Bootstrapping autoregressions with conditional heteroskedasticity of unknown form," Journal of Econometrics, Elsevier, vol. 123(1), pages 89-120, November.
    20. Harvey, David I & Leybourne, Stephen J & Newbold, Paul, 1998. "Tests for Forecast Encompassing," Journal of Business & Economic Statistics, American Statistical Association, vol. 16(2), pages 254-259, April.
    21. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-1084, September.
    22. Clements, Michael P & Hendry, David F, 1996. "Multi-step Estimation for Forecasting," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 58(4), pages 657-684, November.
    23. West, Kenneth D., 2006. "Forecast Evaluation," Handbook of Economic Forecasting, Elsevier.
    24. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    25. Stock, James H. & Watson, Mark W., 1999. "Forecasting inflation," Journal of Monetary Economics, Elsevier, vol. 44(2), pages 293-335, October.
    26. Estrella, Arturo & Hardouvelis, Gikas A, 1991. " The Term Structure as a Predictor of Real Economic Activity," Journal of Finance, American Finance Association, vol. 46(2), pages 555-576, June.
    27. Vuong, Quang H, 1989. "Likelihood Ratio Tests for Model Selection and Non-nested Hypotheses," Econometrica, Econometric Society, vol. 57(2), pages 307-333, March.
    28. Newey, Whitney & West, Kenneth, 2014. "A simple, positive semi-definite, heteroscedasticity and autocorrelation consistent covariance matrix," Applied Econometrics, Publishing House "SINERGIA PRESS", vol. 33(1), pages 125-132.
    29. Massimiliano Marcellino, "undated". "Instability and non-linearity in the EMU," Working Papers 211, IGIER (Innocenzo Gasparini Institute for Economic Research), Bocconi University.
    30. Meese, Richard A & Rogoff, Kenneth, 1988. " Was It Real? The Exchange Rate-Interest Differential Relation over the Modern Floating-Rate Period," Journal of Finance, American Finance Association, vol. 43(4), pages 933-948, September.
    31. Stephen G. Cecchetti, 1995. "Inflation Indicators and Inflation Policy," NBER Chapters,in: NBER Macroeconomics Annual 1995, Volume 10, pages 189-236 National Bureau of Economic Research, Inc.
    32. Hansen, Bruce E., 1992. "Convergence to Stochastic Integrals for Dependent Heterogeneous Processes," Econometric Theory, Cambridge University Press, vol. 8(04), pages 489-500, December.
    33. McCracken, Michael W., 2007. "Asymptotics for out of sample tests of Granger causality," Journal of Econometrics, Elsevier, vol. 140(2), pages 719-752, October.
    34. Arturo Estrella & Anthony P. Rodrigues & Sebastian Schich, 2003. "How Stable is the Predictive Power of the Yield Curve? Evidence from Germany and the United States," The Review of Economics and Statistics, MIT Press, vol. 85(3), pages 629-644, August.
    35. James H. Stock & Mark W.Watson, 2003. "Forecasting Output and Inflation: The Role of Asset Prices," Journal of Economic Literature, American Economic Association, vol. 41(3), pages 788-829, September.
    36. West, Kenneth D & McCracken, Michael W, 1998. "Regression-Based Tests of Predictive Ability," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 817-840, November.
    37. Corradi, Valentina & Swanson, Norman R. & Olivetti, Claudia, 2001. "Predictive ability with cointegrated variables," Journal of Econometrics, Elsevier, vol. 104(2), pages 315-358, September.
    38. Schorfheide, Frank, 2005. "VAR forecasting under misspecification," Journal of Econometrics, Elsevier, vol. 128(1), pages 99-136, September.
    39. Mark, Nelson C, 1995. "Exchange Rates and Fundamentals: Evidence on Long-Horizon Predictability," American Economic Review, American Economic Association, vol. 85(1), pages 201-218, March.
    40. Ericsson, Neil R., 1992. "Parameter constancy, mean square forecast errors, and measuring forecast performance: An exposition, extensions, and illustration," Journal of Policy Modeling, Elsevier, vol. 14(4), pages 465-495, August.
    41. Norman R. Swanson, 2000. "An Out of Sample Test for Granger Causality," Econometric Society World Congress 2000 Contributed Papers 0362, Econometric Society.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. repec:wsi:wschap:9789813148543_0011 is not listed on IDEAS
    2. Nelson C. Mark & Donggyu Sul, 2004. "The Use of Predictive Regressions at Alternative Horizons in Finance and Economics," NBER Technical Working Papers 0298, National Bureau of Economic Research, Inc.
    3. Martin D. D. Evans & Richard K. Lyons, 2017. "Meese-Rogoff Redux: Micro-Based Exchange-Rate Forecasting," World Scientific Book Chapters,in: Studies in Foreign Exchange Economics, chapter 11, pages 457-475 World Scientific Publishing Co. Pte. Ltd..
    4. Boucher, Christophe, 2006. "Stock prices-inflation puzzle and the predictability of stock market returns," Economics Letters, Elsevier, vol. 90(2), pages 205-212, February.
    5. McCracken, Michael W., 2007. "Asymptotics for out of sample tests of Granger causality," Journal of Econometrics, Elsevier, vol. 140(2), pages 719-752, October.
    6. Mehmet Balcilar & Rangan Gupta & Stephen M. Miller, 2012. "The Out-of-Sample Forecasting Performance of Non-Linear Models of Regional Housing Prices in the US," Working Papers 201226, University of Pretoria, Department of Economics.
    7. Orphanides, Athanasios & van Norden, Simon, 2005. "The Reliability of Inflation Forecasts Based on Output Gap Estimates in Real Time," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 37(3), pages 583-601, June.
    8. Rapach, David E. & Wohar, Mark E., 2006. "The out-of-sample forecasting performance of nonlinear models of real exchange rate behavior," International Journal of Forecasting, Elsevier, vol. 22(2), pages 341-361.
    9. Mestre, Ricardo, 2007. "Are survey-based inflation expections in the euro area informative?," Working Paper Series 721, European Central Bank.
    10. Barbara Rossi, 2007. "Expectations hypotheses tests at Long Horizons," Econometrics Journal, Royal Economic Society, vol. 10(3), pages 554-579, November.
    11. Clark, Todd E. & West, Kenneth D., 2006. "Using out-of-sample mean squared prediction errors to test the martingale difference hypothesis," Journal of Econometrics, Elsevier, vol. 135(1-2), pages 155-186.
    12. Todd E. Clark & Kenneth D. West, 2005. "Using Out-of-Sample Mean Squared Prediction Errors to Test the Martingale Difference," NBER Technical Working Papers 0305, National Bureau of Economic Research, Inc.
    13. Bessec Marie & Bouabdallah Othman, 2005. "What Causes The Forecasting Failure of Markov-Switching Models? A Monte Carlo Study," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 9(2), pages 1-24, June.
    14. Clark, Todd E. & McCracken, Michael W., 2005. "The power of tests of predictive ability in the presence of structural breaks," Journal of Econometrics, Elsevier, vol. 124(1), pages 1-31, January.
    15. Clark, Todd E. & McCracken, Michael W., 2006. "The Predictive Content of the Output Gap for Inflation: Resolving In-Sample and Out-of-Sample Evidence," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 38(5), pages 1127-1148, August.
    16. Rapach, David E. & Wohar, Mark E. & Rangvid, Jesper, 2005. "Macro variables and international stock return predictability," International Journal of Forecasting, Elsevier, vol. 21(1), pages 137-166.
    17. O. De Bandt & E. Michaux & C. Bruneau & A. Flageollet, 2007. "Forecasting inflation using economic indicators: the case of France," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 26(1), pages 1-22.
    18. Michael Steiner, 2009. "Predicting premiums for the market, size, value, and momentum factors," Financial Markets and Portfolio Management, Springer;Swiss Society for Financial Market Research, vol. 23(2), pages 137-155, June.
    19. Gabriel Moser & Fabio Rumler & Johann Scharler, 2004. "Forecasting Austrian Inflation," Working Papers 91, Oesterreichische Nationalbank (Austrian Central Bank).
    20. Rapach, David E. & Wohar, Mark E., 2006. "In-sample vs. out-of-sample tests of stock return predictability in the context of data mining," Journal of Empirical Finance, Elsevier, vol. 13(2), pages 231-247, March.
    21. Valentina Corradi & Norman R. Swanson, 2007. "Nonparametric Bootstrap Procedures For Predictive Inference Based On Recursive Estimation Schemes," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 48(1), pages 67-109, February.
    22. Todd E. Clark & Michael W. McCracken, 2002. "Forecast-based model selection in the presence of structural breaks," Research Working Paper RWP 02-05, Federal Reserve Bank of Kansas City.
    23. Nombulelo Gumata, Alain Kabundi and Eliphas Ndou, 2013. "Important Channels of Transmission Monetary Policy Shock in South Africa," Working Papers 375, Economic Research Southern Africa.

    More about this item

    Keywords

    Forecasting;

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:fip:fedkrw:rwp01-14. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Lu Dayrit). General contact details of provider: http://edirc.repec.org/data/frbkcus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.