p-Value Adjustments for Asymptotic Control of the Generalized Familywise Error Rate
This paper introduces a computationally efficient bootstrap procedure for obtaining multiplicity-adjusted p-values in situations where multiple hypotheses are tested simultaneously. This new testing procedure accounts for the mutual dependence of the individual statistics, and is shown under weak conditions to maintain asymptotic control of the generalized familywise error rate. Moreover, the estimated critical values (p-values) obtained via our procedure are less sensitive to the inclusion of true hypotheses and, as a result, our test has greater power to identify false hypotheses even as the collection of hypotheses under test increases in size. Another attractive feature of our test is that it leads naturally to balance among the individual hypotheses under test. This feature is especially attractive in settings where balance is desired but alternative approaches, such as those based on studentization, are difficult or infeasible.
|Date of creation:||Apr 2009|
|Date of revision:|
|Contact details of provider:|| Web page: http://www.vanderbilt.edu/econ/wparchive/index.html|
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Joseph P. Romano & Michael Wolf, 2008. "Balanced Control of Generalized Error Rates," IEW - Working Papers 379, Institute for Empirical Research in Economics - University of Zurich.
- Donald W.K. Andrews & Gustavo Soares, 2007.
"Inference for Parameters Defined by Moment Inequalities Using Generalized Moment Selection,"
Cowles Foundation Discussion Papers
1631, Cowles Foundation for Research in Economics, Yale University.
- Donald W. K. Andrews & Gustavo Soares, 2010. "Inference for Parameters Defined by Moment Inequalities Using Generalized Moment Selection," Econometrica, Econometric Society, vol. 78(1), pages 119-157, 01.
- Joe, Harry, 2006. "Generating random correlation matrices based on partial correlations," Journal of Multivariate Analysis, Elsevier, vol. 97(10), pages 2177-2189, November.
- James G. MacKinnon, 2007. "Bootstrap Hypothesis Testing," Working Papers 1127, Queen's University, Department of Economics.
- Dudoit Sandrine & van der Laan Mark J. & Pollard Katherine S., 2004. "Multiple Testing. Part I. Single-Step Procedures for Control of General Type I Error Rates," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 3(1), pages 1-71, June.
- Romano, Joseph P. & Shaikh, Azeem M. & Wolf, Michael, 2008.
"Formalized Data Snooping Based On Generalized Error Rates,"
Cambridge University Press, vol. 24(02), pages 404-447, April.
- Joseph P & Romano & Azeem M. Shaikh & Michael Wolf, 2005. "Formalized Data Snooping Based on Generalized Error Rates," IEW - Working Papers 259, Institute for Empirical Research in Economics - University of Zurich.
- Joseph P. Romano & Michael Wolf, 2005.
"Exact and Approximate Stepdown Methods for Multiple Hypothesis Testing,"
Journal of the American Statistical Association,
American Statistical Association, vol. 100, pages 94-108, March.
- Joseph Romano & Michael Wolf, 2003. "Exact and approximate stepdown methods for multiple hypothesis testing," Economics Working Papers 727, Department of Economics and Business, Universitat Pompeu Fabra.
- Leslie G. Godfrey, 2005. "Controlling the Overall Significance Level of a Battery of Least Squares Diagnostic Tests," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 67(2), pages 263-279, 04.
- Joseph P. Romano & Michael Wolf, 2005.
"Stepwise Multiple Testing as Formalized Data Snooping,"
Econometric Society, vol. 73(4), pages 1237-1282, 07.
- Joseph P. Romano & Michael Wolf, 2003. "Stepwise Multiple Testing as Formalized Data Snooping," Working Papers 17, Barcelona Graduate School of Economics.
- Joseph P. Romano & Michael Wolf, 2003. "Stepwise multiple testing as formalized data snooping," Economics Working Papers 712, Department of Economics and Business, Universitat Pompeu Fabra.
- Donald W. K. Andrews, 2000. "Inconsistency of the Bootstrap when a Parameter Is on the Boundary of the Parameter Space," Econometrica, Econometric Society, vol. 68(2), pages 399-406, March.
- Sandrine Dudoit & Mark van der Laan & Katherine Pollard, 2004. "Multiple Testing. Part I. Single-Step Procedures for Control of General Type I Error Rates," U.C. Berkeley Division of Biostatistics Working Paper Series 1137, Berkeley Electronic Press.
- Hsu, Po-Hsuan & Hsu, Yu-Chin & Kuan, Chung-Ming, 2010. "Testing the predictive ability of technical analysis using a new stepwise test without data snooping bias," Journal of Empirical Finance, Elsevier, vol. 17(3), pages 471-484, June.
When requesting a correction, please mention this item's handle: RePEc:van:wpaper:0905. See general information about how to correct material in RePEc.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (John P. Conley)
If references are entirely missing, you can add them using this form.