This file is part of IDEAS, which uses RePEc data


[ Papers | Articles | Software | Books | Chapters | Authors | Institutions | JEL Classification | NEP reports | Search | New papers by email | Author registration | Rankings | Volunteers | FAQ | Blog | Help! ]

Stepwise Multiple Testing as Formalized Data Snooping

Author info | Abstract | Publisher info | Download info | Related research | Statistics
Author Info
Joseph P. Romano
Michael Wolf
Abstract

It is common in econometric applications that several hypothesis tests are carried out at the same time. The problem then becomes how to decide which hypotheses to reject, accounting for the multitude of tests. In this paper, we suggest a stepwise multiple testing procedure which asymptotically controls the familywise error rate at a desired level. Compared to related single-step methods, our procedure is more powerful in the sense that it often will reject more false hypotheses. In addition, we advocate the use of studentization when it is feasible. Unlike some stepwise methods, our method implicitly captures the joint dependence structure of the test statistics, which results in increased ability to detect alternative hypotheses. We prove our method asymptotically controls the familywise error rate under minimal assumptions. We present our methodology in the context of comparing several strategies to a common benchmark and deciding which strategies actually beat the benchmark. However, our ideas can easily be extended and/or modi ed to other contexts, such as making inference for the individual regression coecients in a multiple regression framework. Some simulation studies show the improvements of our methods over previous proposals. We also provide an application to a set of real data.

Download Info
To download:

If you experience problems downloading a file, check if you have the proper application to view it first. Information about this may be contained in the File-Format links below. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: http://www.econ.upf.edu/docs/papers/downloads/712.pdf
File Format: application/pdf
File Function: Whole Paper
Download Restriction: no

Publisher Info
Paper provided by Department of Economics and Business, Universitat Pompeu Fabra in its series Economics Working Papers with number 712.

Download reference. The following formats are available: HTML (with abstract), plain text (with abstract), BibTeX, RIS (EndNote, RefMan, ProCite), ReDIF
Length:
Date of creation: Oct 2003
Date of revision:
Handle: RePEc:upf:upfgen:712

Contact details of provider:
Web page: http://www.econ.upf.edu/

For technical questions regarding this item, or to correct its listing, contact: ().

Related research
Keywords: Bootstrap; data snooping; familywise error; multiple testing; step-down method;

Find related papers by JEL classification:
C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: General - - - Hypothesis Testing
C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: General - - - Semiparametric and Nonparametric Methods
C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation and Testing

This paper has been announced in the following NEP Reports:

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
  1. Andrews, Donald W K, 1991. "Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimation," Econometrica, Econometric Society, vol. 59(3), pages 817-58, May. [Downloadable!] (restricted)
    Other versions:
  2. Tae-Hwy Lee & Yong Bao & Burak Saltoglu, 2006. "Evaluating predictive performance of value-at-risk models in emerging markets: a reality check," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 25(2), pages 101-128. [Downloadable!]
  3. Delgado, Miguel A. & Rodriguez-Poo, Juan M. & Wolf, Michael, 2001. "Subsampling inference in cube root asymptotics with an application to Manski's maximum score estimator," Economics Letters, Elsevier, vol. 73(2), pages 241-250, November. [Downloadable!] (restricted)
  4. Halbert White, 2000. "A Reality Check for Data Snooping," Econometrica, Econometric Society, vol. 68(5), pages 1097-1126, September.
  5. Leamer, Edward E, 1983. "Let's Take the Con Out of Econometrics," American Economic Review, American Economic Association, vol. 73(1), pages 31-43, March. [Downloadable!] (restricted)
    Other versions:
  6. Andrews, Donald W K & Monahan, J Christopher, 1992. "An Improved Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimator," Econometrica, Econometric Society, vol. 60(4), pages 953-66, July. [Downloadable!] (restricted)
    Other versions:
  7. Lo, Andrew W & MacKinlay, A Craig, 1990. "Data-Snooping Biases in Tests of Financial Asset Pricing Models," Review of Financial Studies, Oxford University Press for Society for Financial Studies, vol. 3(3), pages 431-67. [Downloadable!] (restricted)
    Other versions:
  8. Gonzalo, Jesus & Wolf, Michael, 2005. "Subsampling inference in threshold autoregressive models," Journal of Econometrics, Elsevier, vol. 127(2), pages 201-224, August. [Downloadable!] (restricted)
    Other versions:
Full references

Statistics
Access and download statistics

Did you know? IDEAS was launched in September 1997.

This page was last updated on 2009-11-27.

This information is provided to you by IDEAS at the Department of Economics, College of Liberal Arts and Sciences, University of Connecticut using RePEc data on a server sponsored by the Society for Economic Dynamics.