p-Value Adjustments for Asymptotic Control of the Generalized Familywise Error Rate
This paper introduces a computationally efficient bootstrap procedure for obtaining multiplicity-adjusted p-values in situations where multiple hypotheses are tested simultaneously. This new testing procedure accounts for the mutual dependence of the individual statistics, and is shown under weak conditions to maintain asymptotic control of the generalized familywise error rate. Moreover, the estimated critical values (p-values) obtained via our procedure are less sensitive to the inclusion of true hypotheses and, as a result, our test has greater power to identify false hypotheses even as the collection of hypotheses under test increases in size. Another attractive feature of our test is that it leads naturally to balance among the individual hypotheses under test. This feature is especially attractive in settings where balance is desired but alternative approaches, such as those based on studentization, are difficult or infeasible.
|Date of creation:||Apr 2009|
|Contact details of provider:|| Web page: http://www.vanderbilt.edu/econ/wparchive/index.html|
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Joseph P. Romano & Michael Wolf, 2005.
"Stepwise Multiple Testing as Formalized Data Snooping,"
Econometric Society, vol. 73(4), pages 1237-1282, 07.
- Joseph P. Romano & Michael Wolf, 2003. "Stepwise multiple testing as formalized data snooping," Economics Working Papers 712, Department of Economics and Business, Universitat Pompeu Fabra.
- Joseph P. Romano & Michael Wolf, 2003. "Stepwise Multiple Testing as Formalized Data Snooping," Working Papers 17, Barcelona Graduate School of Economics.
- Romano, Joseph P. & Shaikh, Azeem M. & Wolf, Michael, 2008. "Formalized Data Snooping Based On Generalized Error Rates," Econometric Theory, Cambridge University Press, vol. 24(02), pages 404-447, April.
- Joseph P & Romano & Azeem M. Shaikh & Michael Wolf, 2005. "Formalized Data Snooping Based on Generalized Error Rates," IEW - Working Papers 259, Institute for Empirical Research in Economics - University of Zurich.
- Donald W. K. Andrews, 2000. "Inconsistency of the Bootstrap when a Parameter Is on the Boundary of the Parameter Space," Econometrica, Econometric Society, vol. 68(2), pages 399-406, March.
- Hansen, Peter Reinhard, 2005. "A Test for Superior Predictive Ability," Journal of Business & Economic Statistics, American Statistical Association, vol. 23, pages 365-380, October.
- Joe, Harry, 2006. "Generating random correlation matrices based on partial correlations," Journal of Multivariate Analysis, Elsevier, vol. 97(10), pages 2177-2189, November.
- Peter Hansen, 2003. "Asymptotic Tests of Composite Hypotheses," Working Papers 2003-09, Brown University, Department of Economics.
- Joseph P. Romano & Michael Wolf, 2005. "Exact and Approximate Stepdown Methods for Multiple Hypothesis Testing," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 94-108, March.
- Joseph Romano & Michael Wolf, 2003. "Exact and approximate stepdown methods for multiple hypothesis testing," Economics Working Papers 727, Department of Economics and Business, Universitat Pompeu Fabra.
- Leslie G. Godfrey, 2005. "Controlling the Overall Significance Level of a Battery of Least Squares Diagnostic Tests," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 67(2), pages 263-279, 04.
- Hsu, Po-Hsuan & Hsu, Yu-Chin & Kuan, Chung-Ming, 2010. "Testing the predictive ability of technical analysis using a new stepwise test without data snooping bias," Journal of Empirical Finance, Elsevier, vol. 17(3), pages 471-484, June. Full references (including those not matched with items on IDEAS)
When requesting a correction, please mention this item's handle: RePEc:van:wpaper:0905. See general information about how to correct material in RePEc.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (John P. Conley)
If references are entirely missing, you can add them using this form.