p-Value Adjustments for Asymptotic Control of the Generalized Familywise Error Rate
This paper introduces a computationally efficient bootstrap procedure for obtaining multiplicity-adjusted p-values in situations where multiple hypotheses are tested simultaneously. This new testing procedure accounts for the mutual dependence of the individual statistics, and is shown under weak conditions to maintain asymptotic control of the generalized familywise error rate. Moreover, the estimated critical values (p-values) obtained via our procedure are less sensitive to the inclusion of true hypotheses and, as a result, our test has greater power to identify false hypotheses even as the collection of hypotheses under test increases in size. Another attractive feature of our test is that it leads naturally to balance among the individual hypotheses under test. This feature is especially attractive in settings where balance is desired but alternative approaches, such as those based on studentization, are difficult or infeasible.
|Date of creation:||Apr 2009|
|Date of revision:|
|Contact details of provider:|| Web page: http://www.vanderbilt.edu/econ/wparchive/index.html|
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Joseph P. Romano & Michael Wolf, 2003.
"Stepwise Multiple Testing as Formalized Data Snooping,"
17, Barcelona Graduate School of Economics.
- Joseph P. Romano & Michael Wolf, 2005. "Stepwise Multiple Testing as Formalized Data Snooping," Econometrica, Econometric Society, vol. 73(4), pages 1237-1282, 07.
- Joseph P. Romano & Michael Wolf, 2003. "Stepwise multiple testing as formalized data snooping," Economics Working Papers 712, Department of Economics and Business, Universitat Pompeu Fabra.
- Donald W. K. Andrews & Gustavo Soares, 2010.
"Inference for Parameters Defined by Moment Inequalities Using Generalized Moment Selection,"
Econometric Society, vol. 78(1), pages 119-157, 01.
- Donald W.K. Andrews & Gustavo Soares, 2007. "Inference for Parameters Defined by Moment Inequalities Using Generalized Moment Selection," Cowles Foundation Discussion Papers 1631, Cowles Foundation for Research in Economics, Yale University.
- Leslie G. Godfrey, 2005. "Controlling the Overall Significance Level of a Battery of Least Squares Diagnostic Tests," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 67(2), pages 263-279, 04.
- Donald W. K. Andrews, 2000. "Inconsistency of the Bootstrap when a Parameter Is on the Boundary of the Parameter Space," Econometrica, Econometric Society, vol. 68(2), pages 399-406, March.
- Joe, Harry, 2006. "Generating random correlation matrices based on partial correlations," Journal of Multivariate Analysis, Elsevier, vol. 97(10), pages 2177-2189, November.
- Hsu, Po-Hsuan & Hsu, Yu-Chin & Kuan, Chung-Ming, 2010. "Testing the predictive ability of technical analysis using a new stepwise test without data snooping bias," Journal of Empirical Finance, Elsevier, vol. 17(3), pages 471-484, June.
- Joseph P. Romano & Michael Wolf, 2005.
"Exact and Approximate Stepdown Methods for Multiple Hypothesis Testing,"
Journal of the American Statistical Association,
American Statistical Association, vol. 100, pages 94-108, March.
- Joseph Romano & Michael Wolf, 2003. "Exact and approximate stepdown methods for multiple hypothesis testing," Economics Working Papers 727, Department of Economics and Business, Universitat Pompeu Fabra.
- Joseph P & Romano & Azeem M. Shaikh & Michael Wolf, 2005.
"Formalized Data Snooping Based on Generalized Error Rates,"
IEW - Working Papers
259, Institute for Empirical Research in Economics - University of Zurich.
- Romano, Joseph P. & Shaikh, Azeem M. & Wolf, Michael, 2008. "Formalized Data Snooping Based On Generalized Error Rates," Econometric Theory, Cambridge University Press, vol. 24(02), pages 404-447, April.
- Dudoit Sandrine & van der Laan Mark J. & Pollard Katherine S., 2004. "Multiple Testing. Part I. Single-Step Procedures for Control of General Type I Error Rates," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 3(1), pages 1-71, June.
- Sandrine Dudoit & Mark van der Laan & Katherine Pollard, 2004. "Multiple Testing. Part I. Single-Step Procedures for Control of General Type I Error Rates," U.C. Berkeley Division of Biostatistics Working Paper Series 1137, Berkeley Electronic Press.
- James G. MacKinnon, 2007. "Bootstrap Hypothesis Testing," Working Papers 1127, Queen's University, Department of Economics.
- Joseph P. Romano & Michael Wolf, 2008. "Balanced Control of Generalized Error Rates," IEW - Working Papers 379, Institute for Empirical Research in Economics - University of Zurich.
When requesting a correction, please mention this item's handle: RePEc:van:wpaper:0905. See general information about how to correct material in RePEc.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (John P. Conley)
If references are entirely missing, you can add them using this form.