IDEAS home Printed from https://ideas.repec.org/a/nat/nathum/v5y2021i7d10.1038_s41562-020-01040-1.html
   My bibliography  Save this article

Competition for priority harms the reliability of science, but reforms can help

Author

Listed:
  • Leonid Tiokhin

    (Eindhoven University of Technology)

  • Minhua Yan

    (Arizona State University
    Arizona State University)

  • Thomas J. H. Morgan

    (Arizona State University
    Arizona State University)

Abstract

Incentives for priority of discovery are hypothesized to harm scientific reliability. Here, we evaluate this hypothesis by developing an evolutionary agent-based model of a competitive scientific process. We find that rewarding priority of discovery causes populations to culturally evolve towards conducting research with smaller samples. This reduces research reliability and the information value of the average study. Increased start-up costs for setting up single studies and increased payoffs for secondary results (also known as scoop protection) attenuate the negative effects of competition. Furthermore, large rewards for negative results promote the evolution of smaller sample sizes. Our results confirm the logical coherence of scoop protection reforms at several journals. Our results also imply that reforms to increase scientific efficiency, such as rapid journal turnaround times, may produce collateral damage by incentivizing lower-quality research; in contrast, reforms that increase start-up costs, such as pre-registration and registered reports, may generate incentives for higher-quality research.

Suggested Citation

  • Leonid Tiokhin & Minhua Yan & Thomas J. H. Morgan, 2021. "Competition for priority harms the reliability of science, but reforms can help," Nature Human Behaviour, Nature, vol. 5(7), pages 857-867, July.
    • Handle: RePEc:nat:nathum:v:5:y:2021:i:7:d:10.1038_s41562-020-01040-1
    DOI: 10.1038/s41562-020-01040-1
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41562-020-01040-1
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1038/s41562-020-01040-1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. John P A Ioannidis, 2005. "Why Most Published Research Findings Are False," PLOS Medicine, Public Library of Science, vol. 2(8), pages 1-1, August.
    2. Azar Ofer H., 2015. "A Model of the Academic Review Process with Informed Authors," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 15(2), pages 865-889, April.
    3. Thomas Dohmen & Armin Falk, 2011. "Performance Pay and Multidimensional Sorting: Productivity, Preferences, and Gender," American Economic Review, American Economic Association, vol. 101(2), pages 556-590, April.
    4. Emanuela Lezzi & Piers Fleming & Daniel John Zizzo, 2015. "Does it matter which effort task you use? A comparison of four effort tasks when agents compete for a prize," Working Paper series, University of East Anglia, Centre for Behavioural and Experimental Social Science (CBESS) 15-05, School of Economics, University of East Anglia, Norwich, UK..
    5. Ofer H. Azar, 2005. "The Review Process in Economics: Is It Too Fast?," Southern Economic Journal, John Wiley & Sons, vol. 72(2), pages 482-491, October.
    6. Kevin Gross & Carl T Bergstrom, 2019. "Contest models highlight inherent inefficiencies of scientific funding competitions," PLOS Biology, Public Library of Science, vol. 17(1), pages 1-15, January.
    7. Catherine Bobtcheff & Jérôme Bolte & Thomas Mariotti, 2017. "Researcher’s Dilemma," Review of Economic Studies, Oxford University Press, vol. 84(3), pages 969-1014.
    8. Joost de Winter & Riender Happee, 2013. "Why Selective Publication of Statistically Significant Results Can Be Effective," PLOS ONE, Public Library of Science, vol. 8(6), pages 1-7, June.
    9. Andrew D Higginson & Marcus R Munafò, 2016. "Current Incentives for Scientists Lead to Underpowered Studies with Erroneous Conclusions," PLOS Biology, Public Library of Science, vol. 14(11), pages 1-14, November.
    10. Christopher Cotton, 2013. "Submission Fees and Response Times in Academic Publishing," American Economic Review, American Economic Association, vol. 103(1), pages 501-509, February.
    11. Ofer H. Azar, 2006. "The Academic Review Process: How Can We Make it More Efficient?," The American Economist, Sage Publications, vol. 50(1), pages 37-50, March.
    12. Emmanuel Dechenaux & Dan Kovenock & Roman Sheremeta, 2015. "A survey of experimental research on contests, all-pay auctions and tournaments," Experimental Economics, Springer;Economic Science Association, vol. 18(4), pages 609-669, December.
    13. Richard McElreath & Paul E Smaldino, 2015. "Replication, Communication, and the Population Dynamics of Scientific Discovery," PLOS ONE, Public Library of Science, vol. 10(8), pages 1-16, August.
    14. John P A Ioannidis, 2014. "How to Make More Published Research True," PLOS Medicine, Public Library of Science, vol. 11(10), pages 1-6, October.
    15. Marcel A L M van Assen & Robbie C M van Aert & Michèle B Nuijten & Jelte M Wicherts, 2014. "Why Publishing Everything Is More Effective than Selective Publishing of Statistically Significant Results," PLOS ONE, Public Library of Science, vol. 9(1), pages 1-5, January.
    16. Brian C. Martinson, 2017. "Give researchers a lifetime word limit," Nature, Nature, vol. 550(7676), pages 303-303, October.
    17. Heintzelman Martin & Nocetti Diego, 2009. "Where Should we Submit our Manuscript? An Analysis of Journal Submission Strategies," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 9(1), pages 1-28, September.
    18. Uri Gneezy & Muriel Niederle & Aldo Rustichini, 2003. "Performance in Competitive Environments: Gender Differences," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 118(3), pages 1049-1074.
    19. Derek Leslie, 2005. "Are Delays in Academic Publishing Necessary?," American Economic Review, American Economic Association, vol. 95(1), pages 407-413, March.
    20. George A. Akerlof & Pascal Michaillat, 2018. "Persistence of false paradigms in low-power sciences," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 115(52), pages 13228-13233, December.
    21. Nosek, Brian A. & Ebersole, Charles R. & DeHaven, Alexander Carl & Mellor, David Thomas, 2018. "The Preregistration Revolution," OSF Preprints 2dxu5, Center for Open Science.
    22. Robert J. Calin-Jageman & Geoff Cumming, 2019. "The New Statistics for Better Science: Ask How Much, How Uncertain, and What Else Is Known," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 271-280, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Lucas Rodriguez Forti & Luiz A. Solino & Judit K. Szabo, 2021. "Trade-off between urgency and reduced editorial capacity affect publication speed in ecological and medical journals during 2020," Palgrave Communications, Palgrave Macmillan, vol. 8(1), pages 1-9, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yuqing Zheng & Harry M. Kaiser, 2016. "Submission Demand In Core Economics Journals: A Panel Study," Economic Inquiry, Western Economic Association International, vol. 54(2), pages 1319-1338, April.
    2. Damien Besancenot & João Ricardo Faria & Franklin G. Mixon, 2017. "Academic Research and the Strategic Interaction of Scholars and Editors: A Two-Stage Game," International Game Theory Review (IGTR), World Scientific Publishing Co. Pte. Ltd., vol. 19(01), pages 1-16, March.
    3. Damien Besancenot & Kim Huynh & Radu Vranceanu, 2011. "A Matching Model of the Academic Publication Market," Journal of Institutional and Theoretical Economics (JITE), Mohr Siebeck, Tübingen, vol. 167(4), pages 708-725, December.
    4. Giuseppe Pernagallo, 2023. "Science in the mist: A model of asymmetric information for the research market," Metroeconomica, Wiley Blackwell, vol. 74(2), pages 390-415, May.
    5. Sascha Baghestanian & Sergey V. Popov, 2018. "On publication, refereeing and working hard," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 51(4), pages 1419-1459, November.
    6. Erin Oldford & John Fiset & Anahit Armenakyan, 2023. "The marginalizing effect of journal submission fees in Accounting and Finance," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(8), pages 4611-4650, August.
    7. Bergemann, Dirk & Ottaviani, Marco, 2021. "Information Markets and Nonmarkets," CEPR Discussion Papers 16459, C.E.P.R. Discussion Papers.
    8. Chen, Zhuoqiong (Charlie) & Ong, David & Sheremeta, Roman M., 2015. "The gender difference in the value of winning," Economics Letters, Elsevier, vol. 137(C), pages 226-229.
    9. Dato, Simon & Nieken, Petra, 2014. "Gender differences in competition and sabotage," Journal of Economic Behavior & Organization, Elsevier, vol. 100(C), pages 64-80.
    10. Curtis R. Price & Roman M. Sheremeta, 2015. "Endowment Origin, Demographic Effects, and Individual Preferences in Contests," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 24(3), pages 597-619, September.
    11. Della Giusta, Marina & Di Girolamo, Amalia, 2018. "Have your cake and eat it too: real effort and risk aversion in schoolchildren," MPRA Paper 89528, University Library of Munich, Germany.
    12. Emmanuel Dechenaux & Dan Kovenock & Roman Sheremeta, 2015. "A survey of experimental research on contests, all-pay auctions and tournaments," Experimental Economics, Springer;Economic Science Association, vol. 18(4), pages 609-669, December.
    13. Zhengyang Bao & Andreas Leibbrandt, 2020. "Tournaments with Safeguards: A Blessing or a Curse for Women?," Monash Economics Working Papers 02-20, Monash University, Department of Economics.
    14. Damien Besancenot & Joao R. Faria & Kim Van Huynh, 2009. "Congestion in academic journals under an impartial selection process," Working Papers halshs-00382585, HAL.
    15. Damien Besancenot & Kim Huynh & Joao Faria, 2012. "Search and research: the influence of editorial boards on journals’ quality," Theory and Decision, Springer, vol. 73(4), pages 687-702, October.
    16. Yaron Azrieli, 2024. "Temporary exclusion in repeated contests," Papers 2401.06257, arXiv.org, revised Mar 2024.
    17. Besancenot Damien & Faria João R. & Huynh Kim V., 2014. "Congestion of Academic Journals Under Papers’ Imperfect Selection," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 14(3), pages 1145-1167, July.
    18. Moizer, Peter, 2009. "Publishing in accounting journals: A fair game?," Accounting, Organizations and Society, Elsevier, vol. 34(2), pages 285-304, February.
    19. Barbos, Andrei, 2013. "Project screening with tiered evaluation," Mathematical Social Sciences, Elsevier, vol. 66(3), pages 293-306.
    20. Carpenter, Jeffrey & Frank, Rachel & Huet-Vaughn, Emiliano, 2018. "Gender differences in interpersonal and intrapersonal competitive behavior," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 77(C), pages 170-176.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:nathum:v:5:y:2021:i:7:d:10.1038_s41562-020-01040-1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.