IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0026895.html

Cooperation between Referees and Authors Increases Peer Review Accuracy

Author

Listed:
  • Jeffrey T Leek
  • Margaret A Taub
  • Fernando J Pineda

Abstract

Peer review is fundamentally a cooperative process between scientists in a community who agree to review each other's work in an unbiased fashion. Peer review is the foundation for decisions concerning publication in journals, awarding of grants, and academic promotion. Here we perform a laboratory study of open and closed peer review based on an online game. We show that when reviewer behavior was made public under open review, reviewers were rewarded for refereeing and formed significantly more cooperative interactions (13% increase in cooperation, P = 0.018). We also show that referees and authors who participated in cooperative interactions had an 11% higher reviewing accuracy rate (P = 0.016). Our results suggest that increasing cooperation in the peer review process can lead to a decreased risk of reviewing errors.

Suggested Citation

  • Jeffrey T Leek & Margaret A Taub & Fernando J Pineda, 2011. "Cooperation between Referees and Authors Increases Peer Review Accuracy," PLOS ONE, Public Library of Science, vol. 6(11), pages 1-11, November.
    • Handle: RePEc:plo:pone00:0026895
    DOI: 10.1371/journal.pone.0026895
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0026895
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0026895&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0026895?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Christine Wennerås & Agnes Wold, 1997. "Nepotism and sexism in peer-review," Nature, Nature, vol. 387(6631), pages 341-343, May.
    2. Bernd Pulverer, 2011. "Peer reviews: some are already public," Nature, Nature, vol. 474(7351), pages 285-285, June.
    3. Anna Dreber & David G. Rand & Drew Fudenberg & Martin A. Nowak, 2008. "Winners don’t punish," Nature, Nature, vol. 452(7185), pages 348-351, March.
    4. Daniel Mietchen, 2011. "Peer reviews: make them public," Nature, Nature, vol. 473(7348), pages 452-452, May.
    5. Bernd Pulverer, 2010. "Transparency showcases strength of peer review," Nature, Nature, vol. 468(7320), pages 29-31, November.
    6. Hidde Ploegh, 2011. "End the wasteful tyranny of reviewer experiments," Nature, Nature, vol. 472(7344), pages 391-391, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Necmi K. Avkiran, 2013. "An empirical investigation of the influence of collaboration in Finance on article impact," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(3), pages 911-925, June.
    2. Qianjin Zong & Yafen Xie & Jiechun Liang, 2020. "Does open peer review improve citation count? Evidence from a propensity score matching analysis of PeerJ," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 607-623, October.
    3. Emilija Stojmenova Duh & Andrej Duh & Uroš Droftina & Tim Kos & Urban Duh & Tanja Simonič Korošak & Dean Korošak, 2019. "Publish-and-Flourish: Using Blockchain Platform to Enable Cooperative Scholarly Communication," Publications, MDPI, vol. 7(2), pages 1-15, May.
    4. Mantas Radzvilas & Francesco De Pretis & William Peden & Daniele Tortoli & Barbara Osimani, 2023. "Incentives for Research Effort: An Evolutionary Model of Publication Markets with Double-Blind and Open Review," Computational Economics, Springer;Society for Computational Economics, vol. 61(4), pages 1433-1476, April.
    5. Follert, Florian & Naumann, Chantal & Thieme, Lutz, 2020. "Between scientific publication and public perception: Some economic remarks on the allocation of time in science," Working Papers of the European Institute for Socioeconomics 34, European Institute for Socioeconomics (EIS), Saarbrücken.
    6. Federico Bianchi & Francisco Grimaldo & Giangiacomo Bravo & Flaminio Squazzoni, 2018. "The peer review game: an agent-based model of scientists facing resource constraints and institutional pressures," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 1401-1420, September.
    7. Gaute Wangen, 2015. "Conflicting Incentives Risk Analysis: A Case Study of the Normative Peer Review Process," Administrative Sciences, MDPI, vol. 5(3), pages 1-23, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Albert Banal-Estañol & Qianshuo Liu & Inés Macho-Stadler & David Pérez-Castrillo, 2021. "Similar-to-me Effects in the Grant Application Process: Applicants, Panelists, and the Likelihood of Obtaining Funds," Working Papers 1289, Barcelona School of Economics.
    2. Alexander Isakov & David Rand, 2012. "The Evolution of Coercive Institutional Punishment," Dynamic Games and Applications, Springer, vol. 2(1), pages 97-109, March.
    3. Long Ma & Xiao Han & Zhesi Shen & Wen-Xu Wang & Zengru Di, 2015. "Efficient Reconstruction of Heterogeneous Networks from Time Series via Compressed Sensing," PLOS ONE, Public Library of Science, vol. 10(11), pages 1-12, November.
    4. van den Besselaar, Peter & Sandström, Ulf, 2015. "Early career grants, performance, and careers: A study on predictive validity of grant decisions," Journal of Informetrics, Elsevier, vol. 9(4), pages 826-838.
    5. Patrícia Martinková & Dan Goldhaber & Elena Erosheva, 2018. "Disparities in ratings of internal and external applicants: A case for model-based inter-rater reliability," PLOS ONE, Public Library of Science, vol. 13(10), pages 1-17, October.
    6. Rockenbach, Bettina & Wolff, Irenaeus, 2009. "Institution design in social dilemmas: How to design if you must?," MPRA Paper 16922, University Library of Munich, Germany.
    7. Dominik Wodarz & Natalia L Komarova, 2013. "Dependence of the Firearm-Related Homicide Rate on Gun Availability: A Mathematical Analysis," PLOS ONE, Public Library of Science, vol. 8(7), pages 1-13, July.
    8. Gaechter, S. & Mengel, F. & Tsakas, E. & Vostroknutov, A., 2013. "Growth and inequality in public good games," Research Memorandum 070, Maastricht University, Graduate School of Business and Economics (GSBE).
    9. Amy Hinsley & William J Sutherland & Alison Johnston, 2017. "Men ask more questions than women at a scientific conference," PLOS ONE, Public Library of Science, vol. 12(10), pages 1-14, October.
    10. Flores, Lucas S. & Han, The Anh, 2024. "Evolution of commitment in the spatial public goods game through institutional incentives," Applied Mathematics and Computation, Elsevier, vol. 473(C).
    11. Wilson, Alistair J. & Wu, Hong, 2017. "At-will relationships: How an option to walk away affects cooperation and efficiency," Games and Economic Behavior, Elsevier, vol. 102(C), pages 487-507.
    12. Pan, Qiuhui & Wang, Lingxiao & Shi, Rongrong & Wang, Huan & He, Mingfeng, 2014. "Spatial modes of cooperation based on bounded rationality," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 415(C), pages 421-427.
    13. van Raan, A. F. J. & van Leeuwen, Th. N., 2002. "Assessment of the scientific basis of interdisciplinary, applied research: Application of bibliometric methods in Nutrition and Food Research," Research Policy, Elsevier, vol. 31(4), pages 611-632, May.
    14. Peter van den Besselaar & Ulf Sandström, 2016. "Gender differences in research performance and its impact on careers: a longitudinal case study," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(1), pages 143-162, January.
    15. repec:dau:papers:123456789/8741 is not listed on IDEAS
    16. Luo-Luo Jiang & Matjaž Perc & Attila Szolnoki, 2013. "If Cooperation Is Likely Punish Mildly: Insights from Economic Experiments Based on the Snowdrift Game," PLOS ONE, Public Library of Science, vol. 8(5), pages 1-7, May.
    17. Ohdaira, Tetsushi, 2017. "Characteristics of the evolution of cooperation by the probabilistic peer-punishment based on the difference of payoff," Chaos, Solitons & Fractals, Elsevier, vol. 95(C), pages 77-83.
    18. Fengyuan Liu & Petter Holme & Matteo Chiesa & Bedoor AlShebli & Talal Rahwan, 2023. "Gender inequality and self-publication are common among academic editors," Nature Human Behaviour, Nature, vol. 7(3), pages 353-364, March.
    19. Madjid Eshaghi Gordji & Gholamreza Askari, 2018. "Hyper-rational choice theory," Papers 1801.10520, arXiv.org, revised Feb 2018.
    20. Marta Vohlídalová, 2021. "Early-Career Women Academics: Between Neoliberalism and Gender Conservatism," Sociological Research Online, , vol. 26(1), pages 27-43, March.
    21. repec:cgr:cgsser:03-11 is not listed on IDEAS
    22. Pat O’Connor & Gemma Irvine, 2020. "Multi-Level State Interventions and Gender Equality in Higher Education Institutions: The Irish Case," Administrative Sciences, MDPI, vol. 10(4), pages 1-21, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0026895. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.