IDEAS home Printed from https://ideas.repec.org/a/plo/pbio00/2000995.html
   My bibliography  Save this article

Current Incentives for Scientists Lead to Underpowered Studies with Erroneous Conclusions

Author

Listed:
  • Andrew D Higginson
  • Marcus R Munafò

Abstract

We can regard the wider incentive structures that operate across science, such as the priority given to novel findings, as an ecosystem within which scientists strive to maximise their fitness (i.e., publication record and career success). Here, we develop an optimality model that predicts the most rational research strategy, in terms of the proportion of research effort spent on seeking novel results rather than on confirmatory studies, and the amount of research effort per exploratory study. We show that, for parameter values derived from the scientific literature, researchers acting to maximise their fitness should spend most of their effort seeking novel results and conduct small studies that have only 10%–40% statistical power. As a result, half of the studies they publish will report erroneous conclusions. Current incentive structures are in conflict with maximising the scientific value of research; we suggest ways that the scientific ecosystem could be improved.

Suggested Citation

  • Andrew D Higginson & Marcus R Munafò, 2016. "Current Incentives for Scientists Lead to Underpowered Studies with Erroneous Conclusions," PLOS Biology, Public Library of Science, vol. 14(11), pages 1-14, November.
  • Handle: RePEc:plo:pbio00:2000995
    DOI: 10.1371/journal.pbio.2000995
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.2000995
    Download Restriction: no

    File URL: https://journals.plos.org/plosbiology/article/file?id=10.1371/journal.pbio.2000995&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pbio.2000995?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. John P A Ioannidis, 2005. "Why Most Published Research Findings Are False," PLOS Medicine, Public Library of Science, vol. 2(8), pages 1-1, August.
    2. Estelle Dumas-Mallet & Katherine Button & Thomas Boraud & Marcus Munafo & François Gonon, 2016. "Replication Validity of Initial Association Studies: A Comparison between Psychiatry, Neurology and Four Somatic Diseases," PLOS ONE, Public Library of Science, vol. 11(6), pages 1-20, June.
    3. Robert W. Levis & Steven Henikoff, 1997. "You read it here first," Nature, Nature, vol. 387(6636), pages 843-843, June.
    4. C. Glenn Begley & Lee M. Ellis, 2012. "Raise standards for preclinical cancer research," Nature, Nature, vol. 483(7391), pages 531-533, March.
    5. Simon H. Friedman & Jens O. M. Karlsson, 1997. "A novel paradigm," Nature, Nature, vol. 385(6616), pages 480-480, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Leonid Tiokhin & Minhua Yan & Thomas J. H. Morgan, 2021. "Competition for priority harms the reliability of science, but reforms can help," Nature Human Behaviour, Nature, vol. 5(7), pages 857-867, July.
    2. Brinkerink, Jasper & De Massis, Alfredo & Kellermanns, Franz, 2022. "One finding is no finding: Toward a replication culture in family business research," Journal of Family Business Strategy, Elsevier, vol. 13(4).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Colin F. Camerer & Anna Dreber & Felix Holzmeister & Teck-Hua Ho & Jürgen Huber & Magnus Johannesson & Michael Kirchler & Gideon Nave & Brian A. Nosek & Thomas Pfeiffer & Adam Altmejd & Nick Buttrick , 2018. "Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015," Nature Human Behaviour, Nature, vol. 2(9), pages 637-644, September.
    2. Bettina Bert & Céline Heinl & Justyna Chmielewska & Franziska Schwarz & Barbara Grune & Andreas Hensel & Matthias Greiner & Gilbert Schönfelder, 2019. "Refining animal research: The Animal Study Registry," PLOS Biology, Public Library of Science, vol. 17(10), pages 1-12, October.
    3. Hajko, Vladimír, 2017. "The failure of Energy-Economy Nexus: A meta-analysis of 104 studies," Energy, Elsevier, vol. 125(C), pages 771-787.
    4. Oliver Braganza, 2020. "A simple model suggesting economically rational sample-size choice drives irreproducibility," PLOS ONE, Public Library of Science, vol. 15(3), pages 1-19, March.
    5. Marlo M Vernon & E Andrew Balas & Shaher Momani, 2018. "Are university rankings useful to improve research? A systematic review," PLOS ONE, Public Library of Science, vol. 13(3), pages 1-15, March.
    6. Mueller-Langer, Frank & Fecher, Benedikt & Harhoff, Dietmar & Wagner, Gert G., 2019. "Replication studies in economics—How many and which papers are chosen for replication, and why?," Research Policy, Elsevier, vol. 48(1), pages 62-83.
    7. Owen, P. Dorian, 2018. "Replication to assess statistical adequacy," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 12, pages 1-16.
    8. Bernhard Voelkl & Lucile Vogt & Emily S Sena & Hanno Würbel, 2018. "Reproducibility of preclinical animal research improves with heterogeneity of study samples," PLOS Biology, Public Library of Science, vol. 16(2), pages 1-13, February.
    9. Koessler, Ann-Kathrin & Page, Lionel & Dulleck, Uwe, 2015. "Promoting pro-social behavior with public statements of good intent," MPRA Paper 80072, University Library of Munich, Germany, revised 24 May 2017.
    10. Richard McElreath & Paul E Smaldino, 2015. "Replication, Communication, and the Population Dynamics of Scientific Discovery," PLOS ONE, Public Library of Science, vol. 10(8), pages 1-16, August.
    11. Ádám Kun, 2018. "Publish and Who Should Perish: You or Science?," Publications, MDPI, vol. 6(2), pages 1-16, April.
    12. Mark D Lindner & Richard K Nakamura, 2015. "Examining the Predictive Validity of NIH Peer Review Scores," PLOS ONE, Public Library of Science, vol. 10(6), pages 1-12, June.
    13. Camerer, Colin & Dreber, Anna & Forsell, Eskil & Ho, Teck-Hua & Huber, Jurgen & Johannesson, Magnus & Kirchler, Michael & Almenberg, Johan & Altmejd, Adam & Chan, Taizan & Heikensten, Emma & Holzmeist, 2016. "Evaluating replicability of laboratory experiments in Economics," MPRA Paper 75461, University Library of Munich, Germany.
    14. Isabelle Bartram & Jonathan M Jeschke, 2019. "Do cancer stem cells exist? A pilot study combining a systematic review with the hierarchy-of-hypotheses approach," PLOS ONE, Public Library of Science, vol. 14(12), pages 1-12, December.
    15. Andrea Saltelli & Monica Fiore, 2020. "From sociology of quantification to ethics of quantification," Palgrave Communications, Palgrave Macmillan, vol. 7(1), pages 1-8, December.
    16. Didier Sornette & Spencer Wheatley & Peter Cauwels, 2019. "The Fair Reward Problem: The Illusion Of Success And How To Solve It," Advances in Complex Systems (ACS), World Scientific Publishing Co. Pte. Ltd., vol. 22(03), pages 1-52, May.
    17. van Aert, Robbie Cornelis Maria, 2018. "Dissertation R.C.M. van Aert," MetaArXiv eqhjd, Center for Open Science.
    18. Estelle Dumas-Mallet & Andy Smith & Thomas Boraud & François Gonon, 2017. "Poor replication validity of biomedical association studies reported by newspapers," PLOS ONE, Public Library of Science, vol. 12(2), pages 1-15, February.
    19. Santiago Sanchez-Pages & Claudia Rodriguez-Ruiz & Enrique Turiegano, 2014. "Facial Masculinity: How the Choice of Measurement Method Enables to Detect Its Influence on Behaviour," PLOS ONE, Public Library of Science, vol. 9(11), pages 1-10, November.
    20. Koessler, Ann-Kathrin & Page, Lionel & Dulleck, Uwe, 2018. "Public Statements of Good Conduct Promote Pro-Social Behavior," EconStor Preprints 180669, ZBW - Leibniz Information Centre for Economics.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pbio00:2000995. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosbiology (email available below). General contact details of provider: https://journals.plos.org/plosbiology/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.