IDEAS home Printed from https://ideas.repec.org/a/bla/jorssb/v80y2018i4p649-679.html
   My bibliography  Save this article

AdaPT: an interactive procedure for multiple testing with side information

Author

Listed:
  • Lihua Lei
  • William Fithian

Abstract

We consider the problem of multiple‐hypothesis testing with generic side information: for each hypothesis Hi we observe both a p‐value pi and some predictor xi encoding contextual information about the hypothesis. For large‐scale problems, adaptively focusing power on the more promising hypotheses (those more likely to yield discoveries) can lead to much more powerful multiple‐testing procedures. We propose a general iterative framework for this problem, the adaptive p‐value thresholding procedure which we call AdaPT, which adaptively estimates a Bayes optimal p‐value rejection threshold and controls the false discovery rate in finite samples. At each iteration of the procedure, the analyst proposes a rejection threshold and observes partially censored p‐values, estimates the false discovery proportion below the threshold and proposes another threshold, until the estimated false discovery proportion is below α. Our procedure is adaptive in an unusually strong sense, permitting the analyst to use any statistical or machine learning method she chooses to estimate the optimal threshold, and to switch between different models at each iteration as information accrues. We demonstrate the favourable performance of AdaPT by comparing it with state of the art methods in five real applications and two simulation studies.

Suggested Citation

  • Lihua Lei & William Fithian, 2018. "AdaPT: an interactive procedure for multiple testing with side information," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 80(4), pages 649-679, September.
  • Handle: RePEc:bla:jorssb:v:80:y:2018:i:4:p:649-679
    DOI: 10.1111/rssb.12274
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/rssb.12274
    Download Restriction: no

    File URL: https://libkey.io/10.1111/rssb.12274?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Nikolaos Ignatiadis & Wolfgang Huber, 2021. "Covariate powered cross‐weighted multiple testing," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 83(4), pages 720-751, September.
    2. Wesley Tansey & Yixin Wang & Raul Rabadan & David Blei, 2020. "Double Empirical Bayes Testing," International Statistical Review, International Statistical Institute, vol. 88(S1), pages 91-113, December.
    3. Zhaoyang Tian & Kun Liang & Pengfei Li, 2021. "A powerful procedure that controls the false discovery rate with directional information," Biometrics, The International Biometric Society, vol. 77(1), pages 212-222, March.
    4. Wang, Jiangzhou & Cui, Tingting & Zhu, Wensheng & Wang, Pengfei, 2023. "Covariate-modulated large-scale multiple testing under dependence," Computational Statistics & Data Analysis, Elsevier, vol. 180(C).
    5. Guillermo Durand & Gilles Blanchard & Pierre Neuvial & Etienne Roquain, 2020. "Post hoc false positive control for structured hypotheses," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 47(4), pages 1114-1148, December.
    6. Otília Menyhart & Boglárka Weltz & Balázs Győrffy, 2021. "MultipleTesting.com: A tool for life science researchers for multiple hypothesis testing correction," PLOS ONE, Public Library of Science, vol. 16(6), pages 1-12, June.
    7. Dennis Leung & Wenguang Sun, 2022. "ZAP: Z$$ Z $$‐value adaptive procedures for false discovery rate control with side information," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 84(5), pages 1886-1946, November.
    8. Tingting Cui & Pengfei Wang & Wensheng Zhu, 2021. "Covariate-adjusted multiple testing in genome-wide association studies via factorial hidden Markov models," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 30(3), pages 737-757, September.
    9. Yang, Chiao-Yu & Lei, Lihua & Ho, Nhat & Fithian, William, 2022. "BONuS: Multiple Multivariate Testing with a Data-Adaptive Test Statistic," Research Papers 4031, Stanford University, Graduate School of Business.
    10. Jelle J Goeman & Aldo Solari, 2024. "On selection and conditioning in multiple testing and selective inference," Biometrika, Biometrika Trust, vol. 111(2), pages 393-416.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssb:v:80:y:2018:i:4:p:649-679. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.