IDEAS home Printed from https://ideas.repec.org/
MyIDEAS: Login to save this paper or follow this series

Controlling the danger of false discoveries in estimating multiple treatment effects

  • Dan Wunderli
Registered author(s):

    I expose the risk of false discoveries in the context of multiple treatment effects. A false discovery is a nonexistent effect that is falsely labeled as statistically significant by its individual t-value. Labeling nonexistent effects as statistically significant has wide-ranging academic and policy-related implications, like costly false conclusions from policy evaluations. I eexamine an empirical labor market model by using state-of-the art multiple testing methods and I provide simulation evidence. By merely using individual t-values at conventional significance levels, the risk of labeling probably nonexistent treatment effects as statistically significant is unacceptably high. Individual t-values even label a number of treatment effects as significant, whereas multiple testing indicates false discoveries in these cases. Tests of a joint null hypothesis such as the well-known F-test control the risk of false discoveries only to a limited extent and do not optimally allow for rejecting individual hypotheses. Multiple testing methods control the risk of false discoveries in general while allowing for individual decisions in the sense of rejecting individual hypotheses.

    If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

    File URL: http://www.econ.uzh.ch/static/wp/econwp060.pdf
    Download Restriction: no

    Paper provided by Department of Economics - University of Zurich in its series ECON - Working Papers with number 060.

    as
    in new window

    Length:
    Date of creation: Jan 2012
    Date of revision:
    Handle: RePEc:zur:econwp:060
    Contact details of provider: Postal: Rämistrasse 71, CH-8006 Zürich
    Phone: +41-1-634 21 37
    Fax: +41-1-634 49 82
    Web page: http://www.econ.uzh.ch/
    Email:


    More information through EDIRC

    References listed on IDEAS
    Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

    as in new window
    1. Lalive, Rafael & van Ours, Jan C. & Zweimüller, Josef, 2002. "The Effect of Benefit Sanctions on the Duration of Unemployment," IZA Discussion Papers 469, Institute for the Study of Labor (IZA).
    2. Jan Boone & Jan C. van Ours, 2000. "Modeling Financial Incentives to Get Unemployed Back to Work," Econometric Society World Congress 2000 Contributed Papers 0973, Econometric Society.
    3. James Heckman & Seong Hyeok Moon & Rodrigo Pinto & Peter Savelyev & Adam Yavitz, 2010. "Analyzing social experiments as implemented: A reexamination of the evidence from the HighScope Perry Preschool Program," Quantitative Economics, Econometric Society, vol. 1(1), pages 1-46, 07.
    4. Joseph P. Romano & Michael Wolf, 2003. "Stepwise Multiple Testing as Formalized Data Snooping," Working Papers 17, Barcelona Graduate School of Economics.
    5. Heckman, James & Singer, Burton, 1984. "A Method for Minimizing the Impact of Distributional Assumptions in Econometric Models for Duration Data," Econometrica, Econometric Society, vol. 52(2), pages 271-320, March.
    Full references (including those not matched with items on IDEAS)

    This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

    When requesting a correction, please mention this item's handle: RePEc:zur:econwp:060. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Marita Kieser)

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If references are entirely missing, you can add them using this form.

    If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.