Forecast evaluation with shared data sets
AbstractData sharing is common practice in forecasting experiments in situations where fresh data samples are difficult or expensive to generate. This means that forecasters often analyze the same data set using a host of different models and sets of explanatory variables. This practice introduces statistical dependencies across forecasting studies that can severely distort statistical inference. Here we examine a new and inexpensive recursive bootstrap procedure that allows forecasters to account explicitly for these dependencies. The procedure allows forecasters to merge empirical evidence and draw inference in the light of previously accumulated results. In an empirical example, we merge results from predictions of daily stock prices based on (1) technical trading rules and (2) calendar rules, demonstrating both the significance of problems arising from data sharing and the simplicity of accounting for data sharing using these new methods.
Download InfoIf you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.
Bibliographic InfoArticle provided by Elsevier in its journal International Journal of Forecasting.
Volume (Year): 19 (2003)
Issue (Month): 2 ()
Contact details of provider:
Web page: http://www.elsevier.com/locate/ijforecast
Other versions of this item:
- C10 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - General
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Diebold, Francis X & Mariano, Roberto S, 1995.
"Comparing Predictive Accuracy,"
Journal of Business & Economic Statistics,
American Statistical Association, vol. 13(3), pages 253-63, July.
- Tom Doan, . "DMARIANO: RATS procedure to compute Diebold-Mariano Forecast Comparison Test," Statistical Software Components RTS00055, Boston College Department of Economics.
- Francis X. Diebold & Robert S. Mariano, 1994. "Comparing Predictive Accuracy," NBER Technical Working Papers 0169, National Bureau of Economic Research, Inc.
- Josef Lakonishok, Seymour Smidt, 1988. "Are Seasonal Anomalies Real? A Ninety-Year Perspective," Review of Financial Studies, Society for Financial Studies, vol. 1(4), pages 403-425.
- Corradi, Valentina & Swanson, Norman R. & Olivetti, Claudia, 2001. "Predictive ability with cointegrated variables," Journal of Econometrics, Elsevier, vol. 104(2), pages 315-358, September.
- Sullivan, Ryan & Timmermann, Allan & White, Halbert, 2001. "Dangers of data mining: The case of calendar effects in stock returns," Journal of Econometrics, Elsevier, vol. 105(1), pages 249-286, November.
- Sullivan, Ryan & Timmermann, Allan G & White, Halbert, 1998.
"Data-Snooping, Technical Trading Rule Performance and the Bootstrap,"
CEPR Discussion Papers
1976, C.E.P.R. Discussion Papers.
- Ryan Sullivan & Allan Timmermann & Halbert White, 1999. "Data-Snooping, Technical Trading Rule Performance, and the Bootstrap," Journal of Finance, American Finance Association, vol. 54(5), pages 1647-1691, October.
- Allan Timmermann & Halbert White & Ryan Sullivan, 1998. "Data-Snooping, Technical Trading, Rule Performance and the Bootstrap," FMG Discussion Papers dp303, Financial Markets Group.
- Andrew W. Lo & A. Craig MacKinlay, 1989.
"Data-Snooping Biases in Tests of Financial Asset Pricing Models,"
NBER Working Papers
3001, National Bureau of Economic Research, Inc.
- Lo, Andrew W & MacKinlay, A Craig, 1990. "Data-Snooping Biases in Tests of Financial Asset Pricing Models," Review of Financial Studies, Society for Financial Studies, vol. 3(3), pages 431-67.
- Lo, Andrew W. (Andrew Wen-Chuan) & MacKinlay, Archie Craig, 1955-, 1989. "Data-snooping biases in tests of financial asset pricing models," Working papers 3020-89., Massachusetts Institute of Technology (MIT), Sloan School of Management.
- Kenneth D. West, 1994.
"Asymptotic Inference About Predictive Ability,"
- Newey, Whitney & West, Kenneth, 2014.
"A simple, positive semi-definite, heteroscedasticity and autocorrelation consistent covariance matrix,"
Publishing House "SINERGIA PRESS", vol. 33(1), pages 125-132.
- Newey, Whitney K & West, Kenneth D, 1987. "A Simple, Positive Semi-definite, Heteroskedasticity and Autocorrelation Consistent Covariance Matrix," Econometrica, Econometric Society, vol. 55(3), pages 703-08, May.
- Whitney K. Newey & Kenneth D. West, 1986. "A Simple, Positive Semi-Definite, Heteroskedasticity and AutocorrelationConsistent Covariance Matrix," NBER Technical Working Papers 0055, National Bureau of Economic Research, Inc.
- Halbert White, 2000. "A Reality Check for Data Snooping," Econometrica, Econometric Society, vol. 68(5), pages 1097-1126, September.
- Hamilton, James D, 1989. "A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle," Econometrica, Econometric Society, vol. 57(2), pages 357-84, March.
- Clements, Michael P. & Franses, Philip Hans & Swanson, Norman R., 2004.
"Forecasting economic and financial time-series with non-linear models,"
International Journal of Forecasting,
Elsevier, vol. 20(2), pages 169-183.
- Michael P. Clements & Philip Hans Franses & Norman R. Swanson, 2003. "Forecasting economic and financial time-series with non-linear models," Departmental Working Papers 200309, Rutgers University, Department of Economics.
- Sasa Zikovic & Randall Filer, 2012.
"Ranking of VaR and ES Models: Performance in Developed and Emerging Markets,"
CESifo Working Paper Series
3980, CESifo Group Munich.
- Saša ŽIKOVIÆ & Randall K. FILER, 2013. "Ranking of VaR and ES Models: Performance in Developed and Emerging Markets," Czech Journal of Economics and Finance (Finance a uver), Charles University Prague, Faculty of Social Sciences, vol. 63(4), pages 327-359, August.
- Lu, Tsung-Hsun, 2014. "The profitability of candlestick charting in the Taiwan stock market," Pacific-Basin Finance Journal, Elsevier, vol. 26(C), pages 65-78.
- Fang, Jiali & Jacobsen, Ben & Qin, Yafeng, 2014. "Predictability of the simple technical trading rules: An out-of-sample test," Review of Financial Economics, Elsevier, vol. 23(1), pages 30-45.
- Gael M. Martin & Andrew Reidy & Jill Wright, 2007.
"Does the Option Market Produce Superior Forecasts of Noise-Corrected Volatility Measures?,"
Monash Econometrics and Business Statistics Working Papers
5/07, Monash University, Department of Econometrics and Business Statistics.
- Gael M. Martin & Andrew Reidy & Jill Wright, 2009. "Does the option market produce superior forecasts of noise-corrected volatility measures?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 24(1), pages 77-104.
- Shynkevich, Andrei, 2012. "Short-term predictability of equity returns along two style dimensions," Journal of Empirical Finance, Elsevier, vol. 19(5), pages 675-685.
- De Gooijer, Jan G. & Hyndman, Rob J., 2006. "25 years of time series forecasting," International Journal of Forecasting, Elsevier, vol. 22(3), pages 443-473.
- Park, Cheol-Ho & Irwin, Scott H., 2004.
"The Profitability Of Technical Trading Rules In Us Futures Markets: A Data Snooping Free Test,"
2004 Conference, April 19-20, 2004, St. Louis, Missouri
19011, NCR-134 Conference on Applied Commodity Price Analysis, Forecasting, and Market Risk Management.
- Park, Cheol-Ho & Irwin, Scott H., 2005. "The Profitability of Technical Trading Rules in US Futures Markets: A Data Snooping Free Test," AgMAS Project Research Reports 14771, University of Illinois at Urbana-Champaign, Department of Agricultural and Consumer Economics.
- Jan G. de Gooijer & Rob J. Hyndman, 2005. "25 Years of IIF Time Series Forecasting: A Selective Review," Tinbergen Institute Discussion Papers 05-068/4, Tinbergen Institute.
- Park, Cheol-Ho & Irwin, Scott H., 2005. "A Reality Check on Technical Trading Rule Profits in US Futures Markets," 2005 Conference, April 18-19, 2005, St. Louis, Missouri 19039, NCR-134 Conference on Applied Commodity Price Analysis, Forecasting, and Market Risk Management.
- Shynkevich, Andrei, 2013. "Time-series momentum as an intra- and inter-industry effect: Implications for market efficiency," Journal of Economics and Business, Elsevier, vol. 69(C), pages 64-85.
- Robert Ślepaczuk & Grzegorz Zakrzewski & Paweł Sakowski, 2012. "Investment strategies beating the market. What can we squeeze from the market?," Working Papers 2012-04, Faculty of Economic Sciences, University of Warsaw.
- Shynkevich, Andrei, 2012. "Performance of technical analysis in growth and small cap segments of the US equity market," Journal of Banking & Finance, Elsevier, vol. 36(1), pages 193-208.
- Cheol-Ho Park & Scott H. Irwin, 2007. "What Do We Know About The Profitability Of Technical Analysis?," Journal of Economic Surveys, Wiley Blackwell, vol. 21(4), pages 786-826, 09.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Zhang, Lei).
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If references are entirely missing, you can add them using this form.
If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.
Please note that corrections may take a couple of weeks to filter through the various RePEc services.