IDEAS home Printed from https://ideas.repec.org/p/zur/econwp/219.html
   My bibliography  Save this paper

Efficient computation of adjusted p-values for resampling-based stepdown multiple testing

Author

Listed:
  • Joseph P. Romano
  • Michael Wolf

Abstract

There has been a recent interest in reporting p-values adjusted for resampling-based stepdown multiple testing procedures proposed in Romano and Wolf (2005a,b). The original papers only describe how to carry out multiple testing at a fixed significance level. Computing adjusted p-values instead in an efficient manner is not entirely trivial. Therefore, this paper fills an apparent gap by detailing such an algorithm.

Suggested Citation

  • Joseph P. Romano & Michael Wolf, 2016. "Efficient computation of adjusted p-values for resampling-based stepdown multiple testing," ECON - Working Papers 219, Department of Economics - University of Zurich.
  • Handle: RePEc:zur:econwp:219
    as

    Download full text from publisher

    File URL: http://www.econ.uzh.ch/static/wp/econwp219.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Joseph P. Romano & Michael Wolf, 2005. "Stepwise Multiple Testing as Formalized Data Snooping," Econometrica, Econometric Society, vol. 73(4), pages 1237-1282, July.
    2. Joseph P. Romano & Michael Wolf, 2005. "Exact and Approximate Stepdown Methods for Multiple Hypothesis Testing," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 94-108, March.
    3. Will Dobbie & Roland G. Fryer Jr., 2015. "The Medium-Term Impacts of High-Achieving Charter Schools," Journal of Political Economy, University of Chicago Press, vol. 123(5), pages 985-1037.
    4. James Heckman & Seong Hyeok Moon & Rodrigo Pinto & Peter Savelyev & Adam Yavitz, 2010. "Analyzing social experiments as implemented: A reexamination of the evidence from the HighScope Perry Preschool Program," Quantitative Economics, Econometric Society, vol. 1(1), pages 1-46, July.
    5. Romano, Joseph P. & Shaikh, Azeem M. & Wolf, Michael, 2008. "Formalized Data Snooping Based On Generalized Error Rates," Econometric Theory, Cambridge University Press, vol. 24(2), pages 404-447, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. John A. List & Azeem M. Shaikh & Yang Xu, 2019. "Multiple hypothesis testing in experimental economics," Experimental Economics, Springer;Economic Science Association, vol. 22(4), pages 773-793, December.
    2. John List & Azeem Shaikh & Atom Vayalinkal, 2021. "Multiple Testing with Covariate Adjustment in Experimental Economics," Natural Field Experiments 00732, The Field Experiments Website.
    3. Rodrigo Pinto & Azeem Shaikh & Adam Yavitz & James Heckman, 2010. "Inference with Imperfect Randomization: The Case of the Perry Preschool Program," 2010 Meeting Papers 1336, Society for Economic Dynamics.
    4. Zeng-Hua Lu, 2019. "Extended MinP Tests of Multiple Hypotheses," Papers 1911.04696, arXiv.org.
    5. Adriano Koshiyama & Nick Firoozye, 2019. "Avoiding Backtesting Overfitting by Covariance-Penalties: an empirical investigation of the ordinary and total least squares cases," Papers 1905.05023, arXiv.org.
    6. Christopher J. Bennett, 2009. "p-Value Adjustments for Asymptotic Control of the Generalized Familywise Error Rate," Vanderbilt University Department of Economics Working Papers 0905, Vanderbilt University Department of Economics.
    7. Sandner, Malte & Cornelissen, Thomas & Jungmann, Tanja & Herrmann, Peggy, 2018. "Evaluating the effects of a targeted home visiting program on maternal and child health outcomes," Journal of Health Economics, Elsevier, vol. 58(C), pages 269-283.
    8. Alexandre Belloni & Victor Chernozhukov & Denis Chetverikov & Christian Hansen & Kengo Kato, 2018. "High-dimensional econometrics and regularized GMM," CeMMAP working papers CWP35/18, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    9. Rute M. Caeiro & Pedro C. Vicente, 2020. "Knowledge of vitamin A deficiency and crop adoption: Evidence from a field experiment in Mozambique," Agricultural Economics, International Association of Agricultural Economists, vol. 51(2), pages 175-190, March.
    10. Orazio Attanasio & Sarah Cattan & Emla Fitzsimons & Costas Meghir & Marta Rubio-Codina, 2020. "Estimating the Production Function for Human Capital: Results from a Randomized Controlled Trial in Colombia," American Economic Review, American Economic Association, vol. 110(1), pages 48-85, January.
    11. Bajgrowicz, Pierre & Scaillet, Olivier, 2012. "Technical trading revisited: False discoveries, persistence tests, and transaction costs," Journal of Financial Economics, Elsevier, vol. 106(3), pages 473-491.
    12. Fabian Kosse & Thomas Deckers & Pia Pinger & Hannah Schildberg-Hörisch & Armin Falk, 2020. "The Formation of Prosociality: Causal Evidence on the Role of Social Environment," Journal of Political Economy, University of Chicago Press, vol. 128(2), pages 434-467.
    13. Rossi, Pauline & Villar, Paola, 2020. "Private health investments under competing risks: Evidence from malaria control in Senegal," Journal of Health Economics, Elsevier, vol. 73(C).
    14. Dolan, Paul & Krekel, Christian & Shreedhar, Ganga & Lee, Helen & Marshall, Claire & Smith, Allison, 2021. "Happy to Help: The Welfare Effects of a Nationwide Micro-Volunteering Programme," IZA Discussion Papers 14431, Institute of Labor Economics (IZA).
    15. Orla Doyle & Nick Fitzpatrick & Judy Lovett & Caroline Rawdon, 2015. "Early intervention and child health: Evidence from a Dublin-based randomized controlled trial," Working Papers 201505, Geary Institute, University College Dublin.
    16. Stange, Jens & Dickhaus, Thorsten & Navarro, Arcadi & Schunk, Daniel, 2016. "Multiplicity- and dependency-adjusted p-values for control of the family-wise error rate," Statistics & Probability Letters, Elsevier, vol. 111(C), pages 32-40.
    17. Hanck, Christoph, 2011. "Now, whose schools are really better (or weaker) than Germany's? A multiple testing approach," Economic Modelling, Elsevier, vol. 28(4), pages 1739-1746, July.
    18. Orla Doyle & Liam Delaney & Christine O'Farrelly & Nick Fitzpatrick & Michael Daly, 2015. "Can Early Intervention Improve Maternal Well-being? Evidence from a Randomized Controlled Trial," Working Papers 2015-015, Human Capital and Economic Opportunity Working Group.
    19. Doyle, Orla & Harmon, Colm & Heckman, James J. & Logue, Caitriona & Moon, Seong Hyeok, 2017. "Early skill formation and the efficiency of parental investment: A randomized controlled trial of home visiting," Labour Economics, Elsevier, vol. 45(C), pages 40-58.
    20. Orazio Attanasio & Sarah Cattan & Emla Fitzsimons & Costas Meghir & Marta Rubio-Codina, 2015. "Estimating the Production Function for Human Capital: Results from a Randomized Control Trial in Colombia," Cowles Foundation Discussion Papers 1987, Cowles Foundation for Research in Economics, Yale University.

    More about this item

    Keywords

    Adjusted p-values; multiple testing; resampling; stepdown procedure;
    All these keywords.

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zur:econwp:219. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/seizhch.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Marita Kieser (email available below). General contact details of provider: https://edirc.repec.org/data/seizhch.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.