IDEAS home Printed from https://ideas.repec.org/a/bla/jorssb/v71y2009i5p1031-1048.html
   My bibliography  Save this article

Controlling the familywise error rate with plug‐in estimator for the proportion of true null hypotheses

Author

Listed:
  • Helmut Finner
  • Veronika Gontscharuk

Abstract

Summary. Estimation of the number or proportion of true null hypotheses in multiple‐testing problems has become an interesting area of research. The first important work in this field was performed by Schweder and Spjøtvoll. Among others, they proposed to use plug‐in estimates for the proportion of true null hypotheses in multiple‐test procedures to improve the power. We investigate the problem of controlling the familywise error rate FWER when such estimators are used as plug‐in estimators in single‐step or step‐down multiple‐test procedures. First we investigate the case of independent p‐values under the null hypotheses and show that a suitable choice of plug‐in estimates leads to control of FWER in single‐step procedures. We also investigate the power and study the asymptotic behaviour of the number of false rejections. Although step‐down procedures are more difficult to handle we briefly consider a possible solution to this problem. Anyhow, plug‐in step‐down procedures are not recommended here. For dependent p‐values we derive a condition for asymptotic control of FWER and provide some simulations with respect to FWER and power for various models and hypotheses.

Suggested Citation

  • Helmut Finner & Veronika Gontscharuk, 2009. "Controlling the familywise error rate with plug‐in estimator for the proportion of true null hypotheses," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 71(5), pages 1031-1048, November.
    • Handle: RePEc:bla:jorssb:v:71:y:2009:i:5:p:1031-1048
    DOI: 10.1111/j.1467-9868.2009.00719.x
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/j.1467-9868.2009.00719.x
    Download Restriction: no
    File URL: https://libkey.io/10.1111/j.1467-9868.2009.00719.x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Sun, Wenguang & Cai, T. Tony, 2007. "Oracle and Adaptive Compound Decision Rules for False Discovery Rate Control," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 901-912, September.
    2. Mette Langaas & Bo Henry Lindqvist & Egil Ferkingstad, 2005. "Estimating the proportion of true null hypotheses, with application to DNA microarray data," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(4), pages 555-572, September.
    3. Jiashun Jin, 2008. "Proportion of non‐zero normal means: universal oracle equivalences and uniformly consistent estimators," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 70(3), pages 461-493, July.
    4. Yoav Benjamini & Yosef Hochberg, 2000. "On the Adaptive Control of the False Discovery Rate in Multiple Testing With Independent Statistics," Journal of Educational and Behavioral Statistics, , vol. 25(1), pages 60-83, March.
    5. M. A. Black, 2004. "A note on the adaptive control of false discovery rates," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 66(2), pages 297-304, May.
    6. Nicolai Meinshausen & Peter Buhlmann, 2005. "Lower bounds for the number of false null hypotheses for multiple testing of associations under general dependence structures," Biometrika, Biometrika Trust, vol. 92(4), pages 893-907, December.
    7. John D. Storey, 2002. "A direct approach to false discovery rates," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 64(3), pages 479-498, August.
    8. Jin, Jiashun & Cai, T. Tony, 2007. "Estimating the Null and the Proportion of Nonnull Effects in Large-Scale Multiple Comparisons," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 495-506, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jules Ellis, 2014. "An Inequality for Correlations in Unidimensional Monotone Latent Variable Models for Binary Variables," Psychometrika, Springer;The Psychometric Society, vol. 79(2), pages 303-316, April.
    2. Marina Bogomolov & Ruth Heller, 2018. "Assessing replicability of findings across two studies of multiple features," Biometrika, Biometrika Trust, vol. 105(3), pages 505-516.
    3. Bryan D. MacGregor & Rainer Schulz & Yuan Zhao, 2021. "Performance and Market Maturity in Mutual Funds: Is Real Estate Different?," The Journal of Real Estate Finance and Economics, Springer, vol. 63(3), pages 437-492, October.
    4. Zhao, Haibing, 2014. "Adaptive FWER control procedure for grouped hypotheses," Statistics & Probability Letters, Elsevier, vol. 95(C), pages 63-70.
    5. Axel Gandy & Georg Hahn, 2016. "A Framework for Monte Carlo based Multiple Testing," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 43(4), pages 1046-1063, December.
    6. Haibing Zhao & Wing Kam Fung, 2018. "Controlling mixed directional false discovery rate in multidimensional decisions with applications to microarray studies," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 27(2), pages 316-337, June.
    7. Dickhaus Thorsten & Straßburger Klaus & Schunk Daniel & Morcillo-Suarez Carlos & Illig Thomas & Navarro Arcadi, 2012. "How to analyze many contingency tables simultaneously in genetic association studies," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 11(4), pages 1-33, July.
    8. Anh-Tuan Hoang & Thorsten Dickhaus, 2022. "On the usage of randomized p-values in the Schweder–Spjøtvoll estimator," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 74(2), pages 289-319, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Rohit Kumar Patra & Bodhisattva Sen, 2016. "Estimation of a two-component mixture model with applications to multiple testing," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 78(4), pages 869-893, September.
    2. Long Qu & Dan Nettleton & Jack C. M. Dekkers, 2012. "Improved Estimation of the Noncentrality Parameter Distribution from a Large Number of t-Statistics, with Applications to False Discovery Rate Estimation in Microarray Data Analysis," Biometrics, The International Biometric Society, vol. 68(4), pages 1178-1187, December.
    3. Chen, Xiongzhi, 2019. "Uniformly consistently estimating the proportion of false null hypotheses via Lebesgue–Stieltjes integral equations," Journal of Multivariate Analysis, Elsevier, vol. 173(C), pages 724-744.
    4. Cipolli III, William & Hanson, Timothy & McLain, Alexander C., 2016. "Bayesian nonparametric multiple testing," Computational Statistics & Data Analysis, Elsevier, vol. 101(C), pages 64-79.
    5. T. Tony Cai & Wenguang Sun, 2017. "Optimal screening and discovery of sparse signals with applications to multistage high throughput studies," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(1), pages 197-223, January.
    6. T. Tony Cai & Wenguang Sun & Weinan Wang, 2019. "Covariate‐assisted ranking and screening for large‐scale two‐sample inference," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 81(2), pages 187-234, April.
    7. Haibing Zhao & Wing Kam Fung, 2018. "Controlling mixed directional false discovery rate in multidimensional decisions with applications to microarray studies," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 27(2), pages 316-337, June.
    8. Li Wang, 2019. "Weighted multiple testing procedure for grouped hypotheses with k-FWER control," Computational Statistics, Springer, vol. 34(2), pages 885-909, June.
    9. Van Hanh Nguyen & Catherine Matias, 2014. "On Efficient Estimators of the Proportion of True Null Hypotheses in a Multiple Testing Setup," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 41(4), pages 1167-1194, December.
    10. Shigeyuki Matsui & Hisashi Noma, 2011. "Estimating Effect Sizes of Differentially Expressed Genes for Power and Sample-Size Assessments in Microarray Experiments," Biometrics, The International Biometric Society, vol. 67(4), pages 1225-1235, December.
    11. Davide Risso & Liam Purvis & Russell B Fletcher & Diya Das & John Ngai & Sandrine Dudoit & Elizabeth Purdom, 2018. "clusterExperiment and RSEC: A Bioconductor package and framework for clustering of single-cell and other large gene expression datasets," PLOS Computational Biology, Public Library of Science, vol. 14(9), pages 1-16, September.
    12. Zhao, Haibing & Fung, Wing Kam, 2016. "A powerful FDR control procedure for multiple hypotheses," Computational Statistics & Data Analysis, Elsevier, vol. 98(C), pages 60-70.
    13. Cai, Qingyun, 2018. "A scoring criterion for rejection of clustered p-values," Computational Statistics & Data Analysis, Elsevier, vol. 121(C), pages 180-189.
    14. Wenguang Sun & T. Tony Cai, 2009. "Large‐scale multiple testing under dependence," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 71(2), pages 393-424, April.
    15. Jiaying Gu & Roger Koenker, 2016. "On a Problem of Robbins," International Statistical Review, International Statistical Institute, vol. 84(2), pages 224-244, August.
    16. David Ruppert & Dan Nettleton & J. T. Gene Hwang, 2007. "Exploring the Information in p-Values for the Analysis and Planning of Multiple-Test Experiments," Biometrics, The International Biometric Society, vol. 63(2), pages 483-495, June.
    17. Jessie Jeng, X., 2016. "Detecting weak signals in high dimensions," Journal of Multivariate Analysis, Elsevier, vol. 147(C), pages 234-246.
    18. Tim Bancroft & Chuanlong Du & Dan Nettleton, 2013. "Estimation of False Discovery Rate Using Sequential Permutation p-Values," Biometrics, The International Biometric Society, vol. 69(1), pages 1-7, March.
    19. Haibing Zhao & Xinping Cui, 2020. "Constructing confidence intervals for selected parameters," Biometrics, The International Biometric Society, vol. 76(4), pages 1098-1108, December.
    20. Habiger, Joshua D. & Peña, Edsel A., 2014. "Compound p-value statistics for multiple testing procedures," Journal of Multivariate Analysis, Elsevier, vol. 126(C), pages 153-166.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssb:v:71:y:2009:i:5:p:1031-1048. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.