IDEAS home Printed from https://ideas.repec.org/a/nat/nathum/v2y2018i1d10.1038_s41562-017-0189-z.html
   My bibliography  Save this article

Redefine statistical significance

Author

Listed:
  • Daniel J. Benjamin

    (University of Southern California)

  • James O. Berger

    (Duke University)

  • Magnus Johannesson

    (Stockholm School of Economics)

  • Brian A. Nosek

    (University of Virginia
    Center for Open Science)

  • E.-J. Wagenmakers

    (University of Amsterdam)

  • Richard Berk

    (University of Pennsylvania
    University of Pennsylvania)

  • Kenneth A. Bollen

    (University of North Carolina Chapel Hill)

  • Björn Brembs

    (Universität Regensburg)

  • Lawrence Brown

    (University of Pennsylvania)

  • Colin Camerer

    (California Institute of Technology)

  • David Cesarini

    (New York University
    The Research Institute of Industrial Economics (IFN))

  • Christopher D. Chambers

    (Cardiff University Brain Research Imaging Centre (CUBRIC))

  • Merlise Clyde

    (Duke University)

  • Thomas D. Cook

    (Northwestern University
    Mathematica Policy Research)

  • Paul De Boeck

    (Ohio State University)

  • Zoltan Dienes

    (University of Sussex)

  • Anna Dreber

    (Stockholm School of Economics)

  • Kenny Easwaran

    (Texas A&M University)

  • Charles Efferson

    (Royal Holloway University of London)

  • Ernst Fehr

    (University of Zurich)

  • Fiona Fidler

    (University of Melbourne)

  • Andy P. Field

    (University of Sussex)

  • Malcolm Forster

    (University of Wisconsin — Madison)

  • Edward I. George

    (University of Pennsylvania)

  • Richard Gonzalez

    (University of Michigan)

  • Steven Goodman

    (Stanford University, General Medical Disciplines)

  • Edwin Green

    (Rutgers University)

  • Donald P. Green

    (Columbia University in the City of New York)

  • Anthony G. Greenwald

    (University of Washington)

  • Jarrod D. Hadfield

    (The University of Edinburgh)

  • Larry V. Hedges

    (Northwestern University)

  • Leonhard Held

    (University of Zurich)

  • Teck Hua Ho

    (National University of Singapore)

  • Herbert Hoijtink

    (Universiteit Utrecht)

  • Daniel J. Hruschka

    (Arizona State University)

  • Kosuke Imai

    (Princeton University)

  • Guido Imbens

    (Stanford University)

  • John P. A. Ioannidis

    (Stanford University)

  • Minjeong Jeon

    (University of California)

  • James Holland Jones

    (Imperial College London
    Department of Earth System Science)

  • Michael Kirchler

    (University of Innsbruck and University of Gothenburg)

  • David Laibson

    (Harvard University)

  • John List

    (University of Chicago)

  • Roderick Little

    (University of Michigan)

  • Arthur Lupia

    (University of Michigan)

  • Edouard Machery

    (University of Pittsburgh)

  • Scott E. Maxwell

    (University of Notre Dame)

  • Michael McCarthy

    (University of Melbourne)

  • Don A. Moore

    (University of California at Berkeley)

  • Stephen L. Morgan

    (Johns Hopkins University)

  • Marcus Munafó

    (University of Bristol
    University of Bristol)

  • Shinichi Nakagawa

    (University of New South Wales)

  • Brendan Nyhan

    (Dartmouth College)

  • Timothy H. Parker

    (Whitman College)

  • Luis Pericchi

    (University of Puerto Rico)

  • Marco Perugini

    (University of Milan-Bicocca)

  • Jeff Rouder

    (University of California)

  • Judith Rousseau

    (Université Paris Dauphine)

  • Victoria Savalei

    (The University of British Columbia)

  • Felix D. Schönbrodt

    (Ludwig-Maximilians-University Munich)

  • Thomas Sellke

    (Purdue University)

  • Betsy Sinclair

    (Washington University in St. Louis)

  • Dustin Tingley

    (Harvard University)

  • Trisha Van Zandt

    (Ohio State University)

  • Simine Vazire

    (University of California)

  • Duncan J. Watts

    (Microsoft Research)

  • Christopher Winship

    (Harvard University)

  • Robert L. Wolpert

    (Duke University)

  • Yu Xie

    (Princeton University)

  • Cristobal Young

    (Stanford University)

  • Jonathan Zinman

    (Dartmouth College)

  • Valen E. Johnson

    (Texas A&M University)

Abstract

We propose to change the default P-value threshold for statistical significance from 0.05 to 0.005 for claims of new discoveries.

Suggested Citation

  • Daniel J. Benjamin & James O. Berger & Magnus Johannesson & Brian A. Nosek & E.-J. Wagenmakers & Richard Berk & Kenneth A. Bollen & Björn Brembs & Lawrence Brown & Colin Camerer & David Cesarini & Chr, 2018. "Redefine statistical significance," Nature Human Behaviour, Nature, vol. 2(1), pages 6-10, January.
    • Handle: RePEc:nat:nathum:v:2:y:2018:i:1:d:10.1038_s41562-017-0189-z
    DOI: 10.1038/s41562-017-0189-z
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41562-017-0189-z
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1038/s41562-017-0189-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    • Daniel Benjamin & James Berger & Magnus Johannesson & Brian Nosek & E. Wagenmakers & Richard Berk & Kenneth Bollen & Bjorn Brembs & Lawrence Brown & Colin Camerer & David Cesarini & Christopher Chambe, 2017. "Redefine Statistical Significance," Artefactual Field Experiments 00612, The Field Experiments Website.

    References listed on IDEAS

    as
    1. Sellke T. & Bayarri M. J. & Berger J. O., 2001. "Calibration of rho Values for Testing Precise Null Hypotheses," The American Statistician, American Statistical Association, vol. 55, pages 62-71, February.
    2. Camerer, Colin & Dreber, Anna & Forsell, Eskil & Ho, Teck-Hua & Huber, Jurgen & Johannesson, Magnus & Kirchler, Michael & Almenberg, Johan & Altmejd, Adam & Chan, Taizan & Heikensten, Emma & Holzmeist, 2016. "Evaluating replicability of laboratory experiments in Economics," MPRA Paper 75461, University Library of Munich, Germany.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jyotirmoy Sarkar, 2018. "Will P†Value Triumph over Abuses and Attacks?," Biostatistics and Biometrics Open Access Journal, Juniper Publishers Inc., vol. 7(4), pages 66-71, July.
    2. Mueller-Langer, Frank & Andreoli-Versbach, Patrick, 2018. "Open access to research data: Strategic delay and the ambiguous welfare effects of mandatory data disclosure," Information Economics and Policy, Elsevier, vol. 42(C), pages 20-34.
    3. Gechert, Sebastian & Mey, Bianka & Opatrny, Matej & Havranek, Tomas & Stanley, T. D. & Bom, Pedro R. D. & Doucouliagos, Hristos & Heimberger, Philipp & Irsova, Zuzana & Rachinger, Heiko J., 2023. "Conventional Wisdom, Meta-Analysis, and Research Revision in Economics," EconStor Preprints 280745, ZBW - Leibniz Information Centre for Economics.
    4. Bigoni, Maria & Camera, Gabriele & Casari, Marco, 2020. "Money is more than memory," Journal of Monetary Economics, Elsevier, vol. 110(C), pages 99-115.
    5. Okunade, Albert & Osmani, Ahmad, 2020. "Effects of life expectancy on economic growth: New results using the flexible Box-Cox power transformation model," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, issue Latest Ar.
    6. Brice Corgnet & Cary Deck & Mark DeSantis & Kyle Hampton & Erik O. Kimbrough, 2023. "When Do Security Markets Aggregate Dispersed Information?," Management Science, INFORMS, vol. 69(6), pages 3697-3729, June.
    7. Lorko, Matej & Servátka, Maroš & Zhang, Le, 2023. "Hidden inefficiency: Strategic inflation of project schedules," Journal of Economic Behavior & Organization, Elsevier, vol. 206(C), pages 313-326.
    8. Albert J. Menkveld & Anna Dreber & Félix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Gunther Capelle-Blancard, 2021. "Non-Standard Errors," Documents de travail du Centre d'Economie de la Sorbonne 21033, Université Panthéon-Sorbonne (Paris 1), Centre d'Economie de la Sorbonne.
      • Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Neusüss, Sebastian & Razen, Michael & Weitzel, Utz & Abad-Díaz, David & Abudy, Mena, 2021. "Non-Standard Errors," Working Papers 2021:17, Lund University, Department of Economics.
      • Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neussüs & Michael Razen & Utz Weitzel & Christian Brownlees & Javier Gil-Bazo, 2021. "Non-Standard Errors," Working Papers 1303, Barcelona School of Economics.
      • Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & David Abad-Díaz & Menachem Abudy & To, 2021. "Non-Standard Errors," Working Paper Series, Social and Economic Sciences 2021-11, Faculty of Social and Economic Sciences, Karl-Franzens-University Graz.
      • Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Jürgen & Johannesson, Magnus & Kirchler, Michael & Neusüss, Sebastian & Razen, Michael & Weitzel, Utz, 2021. "Non-standard errors," IWH Discussion Papers 11/2021, Halle Institute for Economic Research (IWH).
      • Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neussüs & Michael Razen & Utz Weitzel & Christian T. Brownlees & Javier Gil-Baz, 2021. "Non-standard errors," Economics Working Papers 1807, Department of Economics and Business, Universitat Pompeu Fabra.
      • Albert J. et al. Menkveld, 2021. "Non-Standard Errors," CESifo Working Paper Series 9453, CESifo.
      • Albert J Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Gunther Capelle-Blancard & David Abad-Dí, 2021. "Non-Standard Errors," Post-Print halshs-03500882, HAL.
      • Francesco Franzoni & Roxana Mihet & Markus Leippold & Per Ostberg & Olivier Scaillet & Norman Schürhoff & Oksana Bashchenko & Nicola Mano & Michele Pelli, 2022. "Non-Standard Errors," Swiss Finance Institute Research Paper Series 22-09, Swiss Finance Institute.
      • Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Edwin Baidoo & Michael Frömmel & et al, 2021. "Non-Standard Errors," Working Papers of Faculty of Economics and Business Administration, Ghent University, Belgium 21/1032, Ghent University, Faculty of Economics and Business Administration.
      • Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Hasse, Jean-Baptiste & e.a.,, 2023. "Non-Standard Errors," LIDAM Reprints LFIN 2023002, Université catholique de Louvain, Louvain Finance (LFIN).
      • Moinas, Sophie & Declerck, Fany & Menkveld, Albert J. & Dreber, Anna, 2023. "Non-Standard Errors," TSE Working Papers 23-1451, Toulouse School of Economics (TSE).
      • Menkveld, A. & Dreber, A. & Holzmeister, F. & Huber, J. & Johannesson, M. & Kirchler, M. & Neusüss, S. & Razen, M. & Neusüss, S. & Neusüss, S., 2021. "Non-Standard Errors," Cambridge Working Papers in Economics 2182, Faculty of Economics, University of Cambridge.
      • Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Jürgen & Johannesson, Magnus & Kirchler, Michael & Neusüss, Sebastian & Razen, Michael & Weitzel, Utz, 2021. "Non-standard errors," SAFE Working Paper Series 327, Leibniz Institute for Financial Research SAFE.
      • Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Jürgen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & David Abad-Dí­az & Menachem Abudy & Tobi, 2021. "Non-Standard Errors," Working Papers 2021-31, Faculty of Economics and Statistics, Universität Innsbruck.
      • Ferrara, Gerardo & Jurkatis, Simon, 2021. "Non-standard errors," Bank of England working papers 955, Bank of England.
      • Ciril Bosch-Rosa & Bernhard Kassner, 2023. "Non-Standard Errors," Rationality and Competition Discussion Paper Series 385, CRC TRR 190 Rationality and Competition.
      • Albert J Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Gunther Capelle-Blancard & David Abad-Dí, 2021. "Non-Standard Errors," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) halshs-03500882, HAL.
      • Menkveld, A. & Dreber, A. & Holzmeister, F. & Huber, J. & Johannesson, M. & Kirchler, M. & Neusüss, S. & Razen, M. & Neusüss, S. & Neusüss, S., 2021. "Non-Standard Errors," Janeway Institute Working Papers 2112, Faculty of Economics, University of Cambridge.
      • Wolff, Christian & Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Neusüess, Sebastian & Razen, Michael & Weitzel, Utz, 2021. "Non-Standard Errors," CEPR Discussion Papers 16751, C.E.P.R. Discussion Papers.
    9. Colin F. Camerer & Anna Dreber & Felix Holzmeister & Teck-Hua Ho & Jürgen Huber & Magnus Johannesson & Michael Kirchler & Gideon Nave & Brian A. Nosek & Thomas Pfeiffer & Adam Altmejd & Nick Buttrick , 2018. "Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015," Nature Human Behaviour, Nature, vol. 2(9), pages 637-644, September.
    10. Oliver Schilke & Sheen S. Levine & Olenka Kacperczyk & Lynne G. Zucker, 2019. "Call for Papers-Special Issue on Experiments in Organizational Theory," Organization Science, INFORMS, vol. 30(1), pages 232-234, February.
    11. Bachmann, Dirk & Dette, Holger, 2004. "A note on the Bickel-Rosenblatt test in autoregressive time series," Technical Reports 2004,17, Technische Universität Dortmund, Sonderforschungsbereich 475: Komplexitätsreduktion in multivariaten Datenstrukturen.
    12. Özalp Özer & Upender Subramanian & Yu Wang, 2018. "Information Sharing, Advice Provision, or Delegation: What Leads to Higher Trust and Trustworthiness?," Management Science, INFORMS, vol. 64(1), pages 474-493, January.
    13. Adesina, Adedoyin & Akogun, Oladele & Dillon, Andrew & Friedman, Jed & Njobdi, Sani & Serneels, Pieter, 2017. "Robustness and External Validity: What do we Learn from Repeated Study Designs over Time?," 2018 Allied Social Sciences Association (ASSA) Annual Meeting, January 5-7, 2018, Philadelphia, Pennsylvania 266292, Agricultural and Applied Economics Association.
    14. Isaiah Andrews & Maximilian Kasy, 2019. "Identification of and Correction for Publication Bias," American Economic Review, American Economic Association, vol. 109(8), pages 2766-2794, August.
    15. Graham Elliott & Nikolay Kudrin & Kaspar Wüthrich, 2022. "Detecting p‐Hacking," Econometrica, Econometric Society, vol. 90(2), pages 887-906, March.
    16. Aj A Bostian & Christoph Heinzel, 2020. "Robustness of Inferences in Risk and Time Experiments to Lifecycle Asset Integration," Post-Print hal-03358620, HAL.
    17. Burlig, Fiona, 2018. "Improving transparency in observational social science research: A pre-analysis plan approach," Economics Letters, Elsevier, vol. 168(C), pages 56-60.
    18. Galanis, S. & Ioannou, C. & Kotronis, S., 2019. "Information Aggregation Under Ambiguity: Theory and Experimental Evidence," Working Papers 20/05, Department of Economics, City University London.
    19. Maurizio Canavari & Andreas C. Drichoutis & Jayson L. Lusk & Rodolfo M. Nayga, Jr., 2018. "How to run an experimental auction: A review of recent advances," Working Papers 2018-5, Agricultural University of Athens, Department Of Agricultural Economics.
    20. Fuhai Hong & Yohanes E. Riyanto & Ruike Zhang, 2022. "Multidimensional social identity and redistributive preferences: an experimental study," Theory and Decision, Springer, vol. 93(1), pages 151-184, July.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:nathum:v:2:y:2018:i:1:d:10.1038_s41562-017-0189-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.