Advanced Search
MyIDEAS: Login to save this paper or follow this series

Controlling the danger of false discoveries in estimating multiple treatment effects

Contents:

Author Info

  • Dan Wunderli
Registered author(s):

    Abstract

    I expose the risk of false discoveries in the context of multiple treatment effects. A false discovery is a nonexistent effect that is falsely labeled as statistically significant by its individual t-value. Labeling nonexistent effects as statistically significant has wide-ranging academic and policy-related implications, like costly false conclusions from policy evaluations. I eexamine an empirical labor market model by using state-of-the art multiple testing methods and I provide simulation evidence. By merely using individual t-values at conventional significance levels, the risk of labeling probably nonexistent treatment effects as statistically significant is unacceptably high. Individual t-values even label a number of treatment effects as significant, whereas multiple testing indicates false discoveries in these cases. Tests of a joint null hypothesis such as the well-known F-test control the risk of false discoveries only to a limited extent and do not optimally allow for rejecting individual hypotheses. Multiple testing methods control the risk of false discoveries in general while allowing for individual decisions in the sense of rejecting individual hypotheses.

    Download Info

    If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
    File URL: http://www.econ.uzh.ch/static/wp/econwp060.pdf
    Download Restriction: no

    Bibliographic Info

    Paper provided by Department of Economics - University of Zurich in its series ECON - Working Papers with number 060.

    as in new window
    Length:
    Date of creation: Jan 2012
    Date of revision:
    Handle: RePEc:zur:econwp:060

    Contact details of provider:
    Postal: Blümlisalpstrasse 10, CH-8006 Zürich
    Phone: +41-1-634 22 05
    Fax: +41-1-634 49 07
    Email:
    Web page: http://www.econ.uzh.ch/
    More information through EDIRC

    Related research

    Keywords: False discoveries; multiple error rates; multiple treatment effects; labor market;

    Find related papers by JEL classification:

    This paper has been announced in the following NEP Reports:

    References

    References listed on IDEAS
    Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
    as in new window
    1. Rafael Lalive & Jan C. van Ours & Josef Zweimüller, 2005. "The Effect Of Benefit Sanctions On The Duration Of Unemployment," Journal of the European Economic Association, MIT Press, vol. 3(6), pages 1386-1417, December.
    2. James Heckman & Seong Hyeok Moon & Rodrigo Pinto & Peter Savelyev & Adam Yavitz, 2010. "Analyzing social experiments as implemented: A reexamination of the evidence from the HighScope Perry Preschool Program," Quantitative Economics, Econometric Society, vol. 1(1), pages 1-46, 07.
    3. Joseph P. Romano & Michael Wolf, 2003. "Stepwise Multiple Testing as Formalized Data Snooping," Working Papers 17, Barcelona Graduate School of Economics.
    4. Boone, Jan & van Ours, Jan C, 2000. "Modelling Financial Incentives To Get Unemployed Back To Work," CEPR Discussion Papers 2361, C.E.P.R. Discussion Papers.
    5. Heckman, James & Singer, Burton, 1984. "A Method for Minimizing the Impact of Distributional Assumptions in Econometric Models for Duration Data," Econometrica, Econometric Society, vol. 52(2), pages 271-320, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Lists

    This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

    Statistics

    Access and download statistics

    Corrections

    When requesting a correction, please mention this item's handle: RePEc:zur:econwp:060. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Marita Kieser).

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If references are entirely missing, you can add them using this form.

    If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.