Forecast Evaluation with Shared Data Sets
AbstractData sharing is common practice in forecasting experiments in situations where fresh data samples are difficult or expensive to generate. This means that forecasters often analyze the same data set using a host of different models and sets of explanatory variables. This practice introduces statistical dependencies across forecasting studies that can severely distort statistical inference. Here we examine a new and inexpensive recursive bootstrap procedure that allows forecasters to account explicitly for these dependencies. The procedure allows forecasters to merge empirical evidence and draw inference in the light of previously accumulated results. In an empirical example, we merge results from predictions of daily stock prices based on (1) technical trading rules and (2) calendar rules, demonstrating both the significance of problems arising from data sharing and the simplicity of accounting for data sharing using these new methods.
Download InfoIf you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
Bibliographic InfoPaper provided by C.E.P.R. Discussion Papers in its series CEPR Discussion Papers with number 3060.
Date of creation: Nov 2001
Date of revision:
Contact details of provider:
Postal: Centre for Economic Policy Research, 77 Bastwick Street, London EC1V 3PZ
Phone: 44 - 20 - 7183 8801
Fax: 44 - 20 - 7183 8820
Other versions of this item:
- C10 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - General
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Diebold, Francis X & Mariano, Roberto S, 2002.
"Comparing Predictive Accuracy,"
Journal of Business & Economic Statistics,
American Statistical Association, vol. 20(1), pages 134-44, January.
- Hamilton, James D, 1989. "A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle," Econometrica, Econometric Society, vol. 57(2), pages 357-84, March.
- West, Kenneth D, 1996.
"Asymptotic Inference about Predictive Ability,"
Econometric Society, vol. 64(5), pages 1067-84, September.
- Whitney K. Newey & Kenneth D. West, 1986.
"A Simple, Positive Semi-Definite, Heteroskedasticity and AutocorrelationConsistent Covariance Matrix,"
NBER Technical Working Papers
0055, National Bureau of Economic Research, Inc.
- Newey, Whitney K & West, Kenneth D, 1987. "A Simple, Positive Semi-definite, Heteroskedasticity and Autocorrelation Consistent Covariance Matrix," Econometrica, Econometric Society, vol. 55(3), pages 703-08, May.
- Sullivan, Ryan & Timmermann, Allan & White, Halbert, 2001. "Dangers of data mining: The case of calendar effects in stock returns," Journal of Econometrics, Elsevier, vol. 105(1), pages 249-286, November.
- Corradi, Valentina & Swanson, Norman R. & Olivetti, Claudia, 2001. "Predictive ability with cointegrated variables," Journal of Econometrics, Elsevier, vol. 104(2), pages 315-358, September.
- Halbert White, 2000. "A Reality Check for Data Snooping," Econometrica, Econometric Society, vol. 68(5), pages 1097-1126, September.
- Gael M. Martin & Andrew Reidy & Jill Wright, 2009.
"Does the option market produce superior forecasts of noise-corrected volatility measures?,"
Journal of Applied Econometrics,
John Wiley & Sons, Ltd., vol. 24(1), pages 77-104.
- Gael M. Martin & Andrew Reidy & Jill Wright, 2007. "Does the Option Market Produce Superior Forecasts of Noise-Corrected Volatility Measures?," Monash Econometrics and Business Statistics Working Papers 5/07, Monash University, Department of Econometrics and Business Statistics.
- Park, Cheol-Ho & Irwin, Scott H., 2005. "A Reality Check on Technical Trading Rule Profits in US Futures Markets," 2005 Conference, April 18-19, 2005, St. Louis, Missouri 19039, NCR-134 Conference on Applied Commodity Price Analysis, Forecasting, and Market Risk Management.
- Shynkevich, Andrei, 2012. "Performance of technical analysis in growth and small cap segments of the US equity market," Journal of Banking & Finance, Elsevier, vol. 36(1), pages 193-208.
- Park, Cheol-Ho & Irwin, Scott H., 2005.
"The Profitability of Technical Trading Rules in US Futures Markets: A Data Snooping Free Test,"
AgMAS Project Research Reports
14771, University of Illinois at Urbana-Champaign, Department of Agricultural and Consumer Economics.
- Park, Cheol-Ho & Irwin, Scott H., 2004. "The Profitability Of Technical Trading Rules In Us Futures Markets: A Data Snooping Free Test," 2004 Conference, April 19-20, 2004, St. Louis, Missouri 19011, NCR-134 Conference on Applied Commodity Price Analysis, Forecasting, and Market Risk Management.
- repec:dgr:uvatin:2005068 is not listed on IDEAS
- De Gooijer, Jan G. & Hyndman, Rob J., 2006. "25 years of time series forecasting," International Journal of Forecasting, Elsevier, vol. 22(3), pages 443-473.
- Clements, Michael P. & Franses, Philip Hans & Swanson, Norman R., 2004.
"Forecasting economic and financial time-series with non-linear models,"
International Journal of Forecasting,
Elsevier, vol. 20(2), pages 169-183.
- Michael P. Clements & Philip Hans Franses & Norman R. Swanson, 2003. "Forecasting economic and financial time-series with non-linear models," Departmental Working Papers 200309, Rutgers University, Department of Economics.
- Robert Ślepaczuk & Grzegorz Zakrzewski & Paweł Sakowski, 2012. "Investment strategies beating the market. What can we squeeze from the market?," Working Papers 2012-04, Faculty of Economic Sciences, University of Warsaw.
- Saša ŽIKOVIÆ & Randall K. FILER, 2013.
"Ranking of VaR and ES Models: Performance in Developed and Emerging Markets,"
Czech Journal of Economics and Finance (Finance a uver),
Charles University Prague, Faculty of Social Sciences, vol. 63(4), pages 327-359, August.
- Sasa Zikovic & Randall Filer, 2012. "Ranking of VaR and ES Models: Performance in Developed and Emerging Markets," CESifo Working Paper Series 3980, CESifo Group Munich.
- Cheol-Ho Park & Scott H. Irwin, 2007. "What Do We Know About The Profitability Of Technical Analysis?," Journal of Economic Surveys, Wiley Blackwell, vol. 21(4), pages 786-826, 09.
- Shynkevich, Andrei, 2012. "Short-term predictability of equity returns along two style dimensions," Journal of Empirical Finance, Elsevier, vol. 19(5), pages 675-685.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ().
If references are entirely missing, you can add them using this form.