IDEAS home Printed from https://ideas.repec.org/a/eee/stapro/v113y2016icp38-40.html
   My bibliography  Save this article

Efficient computation of adjusted p-values for resampling-based stepdown multiple testing

Author

Listed:
  • Romano, Joseph P.
  • Wolf, Michael

Abstract

There has been a recent interest in reporting p-values adjusted for the resampling-based stepdown multiple testing procedures proposed in Romano and Wolf (2005a,b). The original papers only describe how to carry out multiple testing at a fixed significance level. Computing adjusted p-values instead in an efficient manner is not entirely trivial. Therefore, this paper fills an apparent gap by detailing such an algorithm.

Suggested Citation

  • Romano, Joseph P. & Wolf, Michael, 2016. "Efficient computation of adjusted p-values for resampling-based stepdown multiple testing," Statistics & Probability Letters, Elsevier, vol. 113(C), pages 38-40.
    • Handle: RePEc:eee:stapro:v:113:y:2016:i:c:p:38-40
    DOI: 10.1016/j.spl.2016.02.012
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167715216000389
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.spl.2016.02.012?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. James Heckman & Seong Hyeok Moon & Rodrigo Pinto & Peter Savelyev & Adam Yavitz, 2010. "Analyzing social experiments as implemented: A reexamination of the evidence from the HighScope Perry Preschool Program," Quantitative Economics, Econometric Society, vol. 1(1), pages 1-46, July.
    2. Joseph P. Romano & Michael Wolf, 2005. "Stepwise Multiple Testing as Formalized Data Snooping," Econometrica, Econometric Society, vol. 73(4), pages 1237-1282, July.
    3. James Heckman & Seong Hyeok Moon & Rodrigo Pinto & Peter Savelyev & Adam Yavitz, 2010. "Analyzing social experiments as implemented: evidence from the HighScope Perry Preschool Program," CeMMAP working papers CWP22/10, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    4. Romano, Joseph P. & Shaikh, Azeem M. & Wolf, Michael, 2008. "Formalized Data Snooping Based On Generalized Error Rates," Econometric Theory, Cambridge University Press, vol. 24(2), pages 404-447, April.
    5. Will Dobbie & Roland G. Fryer Jr., 2015. "The Medium-Term Impacts of High-Achieving Charter Schools," Journal of Political Economy, University of Chicago Press, vol. 123(5), pages 985-1037.
    6. Joseph P. Romano & Michael Wolf, 2005. "Exact and Approximate Stepdown Methods for Multiple Hypothesis Testing," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 94-108, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. John A. List & Azeem M. Shaikh & Yang Xu, 2019. "Multiple hypothesis testing in experimental economics," Experimental Economics, Springer;Economic Science Association, vol. 22(4), pages 773-793, December.
    2. Alison Andrew & Orazio Attanasio & Britta Augsburg & Lina Cardona-Sosa & Monimalika Day & Michele Giannola & Sally Grantham-McGregor & Pamela Jervis & Costas Meghir & Marta Rubio-Codina, 2024. "Early Childhood Intervention for the Poor: Long Term Outcomes," NBER Working Papers 32165, National Bureau of Economic Research, Inc.
    3. Sandner, Malte & Cornelissen, Thomas & Jungmann, Tanja & Herrmann, Peggy, 2018. "Evaluating the effects of a targeted home visiting program on maternal and child health outcomes," Journal of Health Economics, Elsevier, vol. 58(C), pages 269-283.
    4. John A. List & Azeem M. Shaikh & Atom Vayalinkal, 2023. "Multiple testing with covariate adjustment in experimental economics," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(6), pages 920-939, September.
    5. Peter A. Savelyev & Kegon T. K. Tan, 2019. "Socioemotional Skills, Education, and Health-Related Outcomes of High-Ability Individuals," American Journal of Health Economics, MIT Press, vol. 5(2), pages 250-280, Spring.
    6. Akabayashi, Hideo & Ruberg, Tim & Shikishima, Chizuru & Yamashita, Jun, 2023. "Education-oriented and care-oriented preschools: Implications on child development," Labour Economics, Elsevier, vol. 84(C).
    7. Orazio Attanasio & Sarah Cattan & Emla Fitzsimons & Costas Meghir & Marta Rubio-Codina, 2020. "Estimating the Production Function for Human Capital: Results from a Randomized Controlled Trial in Colombia," American Economic Review, American Economic Association, vol. 110(1), pages 48-85, January.
    8. Fabian Kosse & Thomas Deckers & Pia Pinger & Hannah Schildberg-Hörisch & Armin Falk, 2020. "The Formation of Prosociality: Causal Evidence on the Role of Social Environment," Journal of Political Economy, University of Chicago Press, vol. 128(2), pages 434-467.
    9. Attanasio, Orazio & Cattan, Sarah & Fitzsimons, Emla & Meghir, Costas & Rubio-Codina, Marta, 2015. "Estimating the Production Function for Human Capital: Results from a Randomized Control Trial in Colombia," IZA Discussion Papers 8856, Institute of Labor Economics (IZA).
    10. Hideo Akabayashi & TIm Ruberg & Chizuru Shikishima & Jun Yamashita, 2023. "Education-Oriented and Care-Oriented Preschools:Implications on Child Development," Keio-IES Discussion Paper Series 2023-009, Institute for Economics Studies, Keio University.
    11. Orla Doyle & Nick Fitzpatrick & Judy Lovett & Caroline Rawdon, 2015. "Early intervention and child health: Evidence from a Dublin-based randomized controlled trial," Working Papers 201505, Geary Institute, University College Dublin.
    12. Santos,Indhira Vanessa & Petroska-Beska,Violeta & Amaro Da Costa Luz Carneiro,Pedro Manuel & Eskreis-Winkler,Lauren & Munoz Boudet,Ana Maria & Berniell,Ines & Krekel,Christian & Arias,Omar & Duckworth, 2021. "Can Grit Be Taught ? Lessons from a Nationwide Field Experiment with Middle-School Students," Policy Research Working Paper Series 9831, The World Bank.
    13. Berger, Eva M. & Fehr, Ernst & Hermes, Henning & Schunk, Daniel & Winkel, Kirsten, 2020. "The Impact of Working Memory Training on Children's Cognitive and Noncognitive Skills," IZA Discussion Papers 13338, Institute of Labor Economics (IZA).
    14. Orla Doyle & Colm Harmon & James J Heckman & Caitriona Logue & Seong Hyeok Moon, 2013. "Measuring Investment in Human Capital Formation: An Experimental Analysis of Early Life Outcomes," Working Papers 201310, School of Economics, University College Dublin.
    15. Rodríguez-Planas, Núria & Sanz-de-Galdeano, Anna & Terskaya, Anastasia, 2018. "Independent Thinking and Hard Working, or Caring and Well Behaved? Short- and Long-Term Impacts of Gender Identity Norms," IZA Discussion Papers 11694, Institute of Labor Economics (IZA).
    16. James J. Heckman & Margaret L. Holland & Kevin K. Makino & Rodrigo Pinto & Maria Rosales-Rueda, 2017. "An Analysis of the Memphis Nurse-Family Partnership Program," NBER Working Papers 23610, National Bureau of Economic Research, Inc.
    17. Doyle, Orla & Harmon, Colm & Heckman, James J. & Logue, Caitriona & Moon, Seong Hyeok, 2017. "Early skill formation and the efficiency of parental investment: A randomized controlled trial of home visiting," Labour Economics, Elsevier, vol. 45(C), pages 40-58.
    18. Pedro Carneiro & Oswald Koussihouèdé & Nathalie Lahire & Costas Meghir & Corina Mommaerts, 2020. "School Grants and Education Quality: Experimental Evidence from Senegal," Economica, London School of Economics and Political Science, vol. 87(345), pages 28-51, January.
    19. Gabriella Conti & Christopher Hansman & James J. Heckman & Matthew F. X. Novak & Angela Ruggiero & Stephen J. Suomi, 2012. "Primate Evidence on the Late Health Effects of Early Life Adversity," Working Papers 2012-008, Human Capital and Economic Opportunity Working Group.
    20. Rodrigo Pinto & Azeem Shaikh & Adam Yavitz & James Heckman, 2010. "Inference with Imperfect Randomization: The Case of the Perry Preschool Program," 2010 Meeting Papers 1336, Society for Economic Dynamics.

    More about this item

    Keywords

    Adjusted p-values; Multiple testing; Resampling; Stepdown procedure;
    All these keywords.

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:stapro:v:113:y:2016:i:c:p:38-40. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/622892/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.