Advanced Search
MyIDEAS: Login to save this paper or follow this series

In-sample or out-of-sample tests of predictability: which one should we use?

Contents:

Author Info

  • Inoue, Atsushi
  • Kilian, Lutz

Abstract

It is widely known that significant in-sample evidence of predictability does not garantuee significant out-of-sample predictability. This is often interpreted as an indiciation that in-sample evidence is likely to be spurious and should be discounted. In this paper we question this conventional wisdom. Our analysis shows that neither data mining nor parameter instability is a plausible explanation of the observed tendency of in-smaple tests to reject the no predictability null more often than out-of-sample tests. We provide an alternative explanation based on the higher power of in-sample tests of predictability. We conclude that results of in-sample tests of predictability will typically be more credible than results of out-of-sample tests. JEL Classification: C12, C22, C52

Download Info

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
File URL: http://www.ecb.europa.eu/pub/pdf/scpwps/ecbwp195.pdf
Download Restriction: no

Bibliographic Info

Paper provided by European Central Bank in its series Working Paper Series with number 0195.

as in new window
Length:
Date of creation: Nov 2002
Date of revision:
Handle: RePEc:ecb:ecbwps:20020195

Contact details of provider:
Postal: Postfach 16 03 19, Frankfurt am Main, Germany
Phone: +49 69 1344 0
Fax: +49 69 1344 6000
Web page: http://www.ecb.europa.eu/home/html/index.en.html
More information through EDIRC

Order Information:
Postal: Press and Information Division, European Central Bank, Kaiserstrasse 29, 60311 Frankfurt am Main, Germany
Email:

Related research

Keywords: Data mining; Out-of-sample inference; Predictability text; Structural change;

Other versions of this item:

Find related papers by JEL classification:

References

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
as in new window
  1. John Y. Campbell & Robert J. Shiller, 1988. "Stock Prices, Earnings and Expected Dividends," NBER Working Papers 2511, National Bureau of Economic Research, Inc.
  2. Kilian, Lutz, 1999. "Exchange Rates and Monetary Fundamentals: What Do We Learn from Long-Horizon Regressions?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., John Wiley & Sons, Ltd., vol. 14(5), pages 491-510, Sept.-Oct.
  3. Richard A. Meese & Andrew K. Rose, 1989. "An empirical assessment of non-linearities in models of exchange rate determination," International Finance Discussion Papers, Board of Governors of the Federal Reserve System (U.S.) 367, Board of Governors of the Federal Reserve System (U.S.).
  4. Richard Clarida & Lucio Sarno & Mark Taylor & Giorgio Valente, 2001. "The Out-of-Sample Success of Term Structure Models as Exchange Rate Predictors: A Step Beyond," NBER Working Papers 8601, National Bureau of Economic Research, Inc.
  5. McCracken, Michael W., 2007. "Asymptotics for out of sample tests of Granger causality," Journal of Econometrics, Elsevier, Elsevier, vol. 140(2), pages 719-752, October.
  6. Inoue, Atsushi & Kilian, Lutz, 2006. "On the selection of forecasting models," Journal of Econometrics, Elsevier, Elsevier, vol. 130(2), pages 273-306, February.
  7. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, American Statistical Association, vol. 20(1), pages 134-44, January.
  8. Meese, Richard A. & Rogoff, Kenneth, 1983. "Empirical exchange rate models of the seventies : Do they fit out of sample?," Journal of International Economics, Elsevier, vol. 14(1-2), pages 3-24, February.
  9. Stock, James H. & Watson, Mark W., 1999. "Forecasting inflation," Journal of Monetary Economics, Elsevier, Elsevier, vol. 44(2), pages 293-335, October.
  10. Lutz Kilian & Mark P. Taylor, 2001. "Why is it so difficult to beat the Random Walk Forecast of Exchange Rates?," Tinbergen Institute Discussion Papers 01-031/4, Tinbergen Institute.
  11. Kenneth D. West, 1994. "Asymptotic Inference About Predictive Ability," Macroeconomics, EconWPA 9410002, EconWPA.
  12. Rossi, Barbara, 2002. "Optimal Tests for Nested Model Selection with Underlying Parameter Instability," Working Papers, Duke University, Department of Economics 02-05, Duke University, Department of Economics.
  13. Todd E. Clark & Michael W. McCracken, 2002. "Forecast-based model selection in the presence of structural breaks," Research Working Paper, Federal Reserve Bank of Kansas City RWP 02-05, Federal Reserve Bank of Kansas City.
  14. Jeremy Berkowitz & Lorenzo Giorgianni, 1996. "Long-horizon exchange rate predictability?," Finance and Economics Discussion Series, Board of Governors of the Federal Reserve System (U.S.) 96-39, Board of Governors of the Federal Reserve System (U.S.).
  15. Berben, R-P. & van Dijk, D.J.C., 1998. "Does the absence of cointegration explain the typical findings in long horizon regressions?," Econometric Institute Research Papers EI 9814, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
  16. Meese, Richard A & Rogoff, Kenneth, 1988. " Was It Real? The Exchange Rate-Interest Differential Relation over the Modern Floating-Rate Period," Journal of Finance, American Finance Association, American Finance Association, vol. 43(4), pages 933-48, September.
  17. Ghysels, E & Hall, A., 1988. "A Test For Structural Stability Of Euler Conditions Parameters Estimated Via The Generalized Methods Of Moments Estimators," Cahiers de recherche, Centre interuniversitaire de recherche en économie quantitative, CIREQ 8837, Centre interuniversitaire de recherche en économie quantitative, CIREQ.
  18. James H. Stock & Mark W.Watson, 2003. "Forecasting Output and Inflation: The Role of Asset Prices," Journal of Economic Literature, American Economic Association, vol. 41(3), pages 788-829, September.
  19. Philippe Jorion & William N. Goetzmann, 1998. "A Longer Look at Dividend Yields," Yale School of Management Working Papers, Yale School of Management ysm41, Yale School of Management.
  20. Todd E. Clark, 2000. "Can out-of-sample forecast comparisons help prevent overfitting?," Research Working Paper, Federal Reserve Bank of Kansas City RWP 00-05, Federal Reserve Bank of Kansas City.
  21. Corradi, V. & Swanson, N.R., 2000. "A Consistent Test for Nonlinear Out of Sample Predictive Accuracy," Discussion Papers 0012, Exeter University, Department of Economics.
  22. Todd E. Clark & Michael W. McCracken, 2000. "Tests of Equal Forecast Accuracy and Encompassing for Nested Models," Econometric Society World Congress 2000 Contributed Papers 0319, Econometric Society.
  23. John Y. Campbell, Robert J. Shiller, 1988. "The Dividend-Price Ratio and Expectations of Future Dividends and Discount Factors," Review of Financial Studies, Society for Financial Studies, vol. 1(3), pages 195-228.
  24. Goetzmann, W.N., 1990. "Testing The Predictive Power Of Dividend Yields," Papers fb-_90-12, Columbia - Graduate School of Business.
  25. Richard H. Clarida & Mark P. Taylor, 1997. "The Term Structure Of Forward Exchange Premiums And The Forecastability Of Spot Exchange Rates: Correcting The Errors," The Review of Economics and Statistics, MIT Press, vol. 79(3), pages 353-361, August.
  26. Corradi, Valentina & Swanson, Norman R. & Olivetti, Claudia, 2001. "Predictive ability with cointegrated variables," Journal of Econometrics, Elsevier, Elsevier, vol. 104(2), pages 315-358, September.
  27. Jon Faust & John H. Rogers & Jonathan H. Wright, 2001. "Exchange rate forecasting: the errors we've really made," International Finance Discussion Papers, Board of Governors of the Federal Reserve System (U.S.) 714, Board of Governors of the Federal Reserve System (U.S.).
  28. Norman R. Swanson & Jeffery D. Amato, 2000. "The real-time predictive content of money for output," BIS Working Papers 96, Bank for International Settlements.
  29. Ashley, R & Granger, C W J & Schmalensee, R, 1980. "Advertising and Aggregate Consumption: An Analysis of Causality," Econometrica, Econometric Society, Econometric Society, vol. 48(5), pages 1149-67, July.
  30. Norman R. Swanson, 2000. "An Out of Sample Test for Granger Causality," Econometric Society World Congress 2000 Contributed Papers 0362, Econometric Society.
  31. Chinn, Menzie D. & Meese, Richard A., 1995. "Banking on currency forecasts: How predictable is change in money?," Journal of International Economics, Elsevier, vol. 38(1-2), pages 161-178, February.
  32. Mark, Nelson C, 1995. "Exchange Rates and Fundamentals: Evidence on Long-Horizon Predictability," American Economic Review, American Economic Association, vol. 85(1), pages 201-18, March.
  33. repec:fth:erroem:9814/a is not listed on IDEAS
  34. Harvey, David I & Leybourne, Stephen J & Newbold, Paul, 1998. "Tests for Forecast Encompassing," Journal of Business & Economic Statistics, American Statistical Association, American Statistical Association, vol. 16(2), pages 254-59, April.
  35. Hansen, Bruce E., 1992. "Convergence to Stochastic Integrals for Dependent Heterogeneous Processes," Econometric Theory, Cambridge University Press, vol. 8(04), pages 489-500, December.
  36. Halbert White, 2000. "A Reality Check for Data Snooping," Econometrica, Econometric Society, Econometric Society, vol. 68(5), pages 1097-1126, September.
  37. Harvey, David & Leybourne, Stephen & Newbold, Paul, 1997. "Testing the equality of prediction mean squared errors," International Journal of Forecasting, Elsevier, Elsevier, vol. 13(2), pages 281-291, June.
  38. Sullivan, Ryan & Timmermann, Allan & White, Halbert, 2001. "Dangers of data mining: The case of calendar effects in stock returns," Journal of Econometrics, Elsevier, Elsevier, vol. 105(1), pages 249-286, November.
  39. repec:cup:etheor:v:8:y:1992:i:4:p:489-500 is not listed on IDEAS
  40. Chao, John & Corradi, Valentina & Swanson, Norman R., 2001. "Out-Of-Sample Tests For Granger Causality," Macroeconomic Dynamics, Cambridge University Press, Cambridge University Press, vol. 5(04), pages 598-620, September.
  41. Fama, Eugene F. & French, Kenneth R., 1988. "Dividend yields and expected stock returns," Journal of Financial Economics, Elsevier, Elsevier, vol. 22(1), pages 3-25, October.
Full references (including those not matched with items on IDEAS)

Citations

Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
as in new window

Cited by:
This item has more than 25 citations. To prevent cluttering this page, these citations are listed on a separate page.

Lists

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

Statistics

Access and download statistics

Corrections

When requesting a correction, please mention this item's handle: RePEc:ecb:ecbwps:20020195. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Official Publications).

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.