IDEAS home Printed from https://ideas.repec.org/a/bpj/ijbist/v12y2016i1p21-29n10.html
   My bibliography  Save this article

Evaluations of the Optimal Discovery Procedure for Multiple Testing

Author

Listed:
  • Rubin Daniel B.

    (U.S. Food and Drug Administration, Silver Spring, MD, USA)

Abstract

The Optimal Discovery Procedure (ODP) is a method for simultaneous hypothesis testing that attempts to gain power relative to more standard techniques by exploiting multivariate structure [1]. Specializing to the example of testing whether components of a Gaussian mean vector are zero, we compare the power of the ODP to a Bonferroni-style method and to the Benjamini-Hochberg method when the testing procedures aim to respectively control certain Type I error rate measures, such as the expected number of false positives or the false discovery rate. We show through theoretical results, numerical comparisons, and two microarray examples that when the rejection regions for the ODP test statistics are chosen such that the procedure is guaranteed to uniformly control a Type I error rate measure, the technique is generally less powerful than competing methods. We contrast and explain these results in light of previously proven optimality theory for the ODP. We also compare the ordering given by the ODP test statistics to the standard rankings based on sorting univariate p-values from smallest to largest. In the cases we considered the standard ordering was superior, and ODP rankings were adversely impacted by correlation.

Suggested Citation

  • Rubin Daniel B., 2016. "Evaluations of the Optimal Discovery Procedure for Multiple Testing," The International Journal of Biostatistics, De Gruyter, vol. 12(1), pages 21-29, May.
    • Handle: RePEc:bpj:ijbist:v:12:y:2016:i:1:p:21-29:n:10
    DOI: 10.1515/ijb-2015-0027
    as

    Download full text from publisher

    File URL: https://doi.org/10.1515/ijb-2015-0027
    Download Restriction: For access to full text, subscription to the journal or payment for the individual article is required.
    File URL: https://libkey.io/10.1515/ijb-2015-0027?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. John D. Storey, 2007. "The optimal discovery procedure: a new approach to simultaneous significance testing," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 69(3), pages 347-368, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Nixon, John H., 2012. "Investigations into refinements of Storey’s method of multiple hypothesis testing minimising the FDR, and its application to test binomial data," Computational Statistics & Data Analysis, Elsevier, vol. 56(12), pages 4381-4398.
    2. Daniel Yekutieli, 2015. "Bayesian tests for composite alternative hypotheses in cross-tabulated data," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 24(2), pages 287-301, June.
    3. Dazard, Jean-Eudes & Sunil Rao, J., 2012. "Joint adaptive mean–variance regularization and variance stabilization of high dimensional data," Computational Statistics & Data Analysis, Elsevier, vol. 56(7), pages 2317-2333.
    4. Luis G. León-Novelo & Peter Müller & Wadih Arap & Mikhail Kolonin & Jessica Sun & Renata Pasqualini & Kim-Anh Do, 2013. "Semiparametric Bayesian Inference for Phage Display Data," Biometrics, The International Biometric Society, vol. 69(1), pages 174-183, March.
    5. Leek Jeffrey T & Storey John D., 2011. "The Joint Null Criterion for Multiple Hypothesis Tests," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 10(1), pages 1-22, June.
    6. Wenguang Sun & T. Tony Cai, 2009. "Large‐scale multiple testing under dependence," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 71(2), pages 393-424, April.
    7. Saharon Rosset & Ruth Heller & Amichai Painsky & Ehud Aharoni, 2022. "Optimal and maximin procedures for multiple testing problems," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 84(4), pages 1105-1128, September.
    8. Habiger, Joshua D. & Peña, Edsel A., 2014. "Compound p-value statistics for multiple testing procedures," Journal of Multivariate Analysis, Elsevier, vol. 126(C), pages 153-166.
    9. Shonosuke Sugasawa & Hisashi Noma, 2021. "Efficient screening of predictive biomarkers for individual treatment selection," Biometrics, The International Biometric Society, vol. 77(1), pages 249-257, March.
    10. Ruth Heller & Saharon Rosset, 2021. "Optimal control of false discovery criteria in the two‐group model," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 83(1), pages 133-155, February.
    11. Huixia Wang & Xuming He, 2008. "An Enhanced Quantile Approach for Assessing Differential Gene Expressions," Biometrics, The International Biometric Society, vol. 64(2), pages 449-457, June.
    12. Edsel Peña & Joshua Habiger & Wensong Wu, 2015. "Classes of multiple decision functions strongly controlling FWER and FDR," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 78(5), pages 563-595, July.
    13. Xiaoquan Wen, 2017. "Robust Bayesian FDR Control Using Bayes Factors, with Applications to Multi-tissue eQTL Discovery," Statistics in Biosciences, Springer;International Chinese Statistical Association, vol. 9(1), pages 28-49, June.
    14. Chen, Xiongzhi, 2019. "Uniformly consistently estimating the proportion of false null hypotheses via Lebesgue–Stieltjes integral equations," Journal of Multivariate Analysis, Elsevier, vol. 173(C), pages 724-744.
    15. Michele Guindani & Peter Müller & Song Zhang, 2009. "A Bayesian discovery procedure," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 71(5), pages 905-925, November.
    16. David Amar & Ron Shamir & Daniel Yekutieli, 2017. "Extracting replicable associations across multiple studies: Empirical Bayes algorithms for controlling the false discovery rate," PLOS Computational Biology, Public Library of Science, vol. 13(8), pages 1-22, August.
    17. Shigeyuki Matsui & Hisashi Noma & Pingping Qu & Yoshio Sakai & Kota Matsui & Christoph Heuck & John Crowley, 2018. "Multi†subgroup gene screening using semi†parametric hierarchical mixture models and the optimal discovery procedure: Application to a randomized clinical trial in multiple myeloma," Biometrics, The International Biometric Society, vol. 74(1), pages 313-320, March.
    18. Hwang J.T. Gene & Liu Peng, 2010. "Optimal Tests Shrinking Both Means and Variances Applicable to Microarray Data Analysis," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 9(1), pages 1-35, October.
    19. Rossell David & Guerra Rudy & Scott Clayton, 2008. "Semi-Parametric Differential Expression Analysis via Partial Mixture Estimation," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 7(1), pages 1-29, April.
    20. Jules Ellis, 2014. "An Inequality for Correlations in Unidimensional Monotone Latent Variable Models for Binary Variables," Psychometrika, Springer;The Psychometric Society, vol. 79(2), pages 303-316, April.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bpj:ijbist:v:12:y:2016:i:1:p:21-29:n:10. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Peter Golla (email available below). General contact details of provider: https://www.degruyter.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.