IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v126y2021i12d10.1007_s11192-021-04182-9.html
   My bibliography  Save this article

The influence of opening up peer review on the citations of journal articles

Author

Listed:
  • Jue Ni

    (Nanjing University)

  • Zhenyue Zhao

    (Nanjing University)

  • Yupo Shao

    (Nanjing University)

  • Shuo Liu

    (Nanjing University)

  • Wanlin Li

    (Nanjing University)

  • Yaoze Zhuang

    (Nanjing University)

  • Junmo Qu

    (Yangzhou University)

  • Yu Cao

    (Nanjing University)

  • Nayuan Lian

    (Nanjing University)

  • Jiang Li

    (Nanjing University)

Abstract

This paper studied whether opening up review reports benefits science in terms of citations by taking Nature Communications as an example. To address this question, we collected the bibliographic records of 7614 papers published by Nature Communications in 2016 and 2017 from the Web of Science database and the disclosed reviewers’ comments and authors’ responses of a subset of 2293 papers. Using a linear regression model, we found no evidence of a citation advantage for the articles which disclosed their peer review documents. We concluded that opening peer review reports did not benefit papers in Nature Communications in terms of citations. We further examined whether the length of the comments and the number of rounds of the review process are associated with the papers’ citations. We found no evidence that the number of rounds is associated with the citations of the articles in Nature Communications. However, longer comments are associated with fewer citations, although the effect is weak.

Suggested Citation

  • Jue Ni & Zhenyue Zhao & Yupo Shao & Shuo Liu & Wanlin Li & Yaoze Zhuang & Junmo Qu & Yu Cao & Nayuan Lian & Jiang Li, 2021. "The influence of opening up peer review on the citations of journal articles," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(12), pages 9393-9404, December.
    • Handle: RePEc:spr:scient:v:126:y:2021:i:12:d:10.1007_s11192-021-04182-9
    DOI: 10.1007/s11192-021-04182-9
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-021-04182-9
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-021-04182-9?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Terrence A. Brooks, 1986. "Evidence of complex citer motivations," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 37(1), pages 34-36, January.
    2. Lutz Bornmann & Rüdiger Mutz & Hans-Dieter Daniel, 2010. "A Reliability-Generalization Study of Journal Peer Reviews: A Multilevel Meta-Analysis of Inter-Rater Reliability and Its Determinants," PLOS ONE, Public Library of Science, vol. 5(12), pages 1-10, December.
    3. Flaminio Squazzoni & Elise Brezis & Ana Marušić, 2017. "Scientometrics of peer review," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 501-502, October.
    4. Ma, Chao & Li, Yiwei & Guo, Feng & Si, Kao, 2019. "The citation trap: Papers published at year-end receive systematically fewer citations," Journal of Economic Behavior & Organization, Elsevier, vol. 166(C), pages 667-687.
    5. Xianwen Wang & Chen Liu & Wenli Mao & Zhichao Fang, 2015. "Erratum to: The open access advantage considering citation, article usage and social media attention," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(3), pages 1149-1149, June.
    6. Xianwen Wang & Chen Liu & Wenli Mao & Zhichao Fang, 2015. "The open access advantage considering citation, article usage and social media attention," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(2), pages 555-564, May.
    7. Lutz Bornmann & Werner Marx & Hermann Schier & Andreas Thor & Hans-Dieter Daniel, 2010. "From black box to white box at open access journals: predictive validity of manuscript reviewing and editorial decisions at Atmospheric Chemistry and Physics," Research Evaluation, Oxford University Press, vol. 19(2), pages 105-118, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sun, Zhuanlan, 2024. "Textual features of peer review predict top-cited papers: An interpretable machine learning perspective," Journal of Informetrics, Elsevier, vol. 18(2).
    2. Cheng, Xi & Wang, Haoran & Tang, Li & Jiang, Weiyan & Zhou, Maotian & Wang, Guoyan, 2024. "Open peer review correlates with altmetrics but not with citations: Evidence from Nature Communications and PLoS One," Journal of Informetrics, Elsevier, vol. 18(3).
    3. Weixi Xie & Pengfei Jia & Guangyao Zhang & Xianwen Wang, 2024. "Are reviewer scores consistent with citations?," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(8), pages 4721-4740, August.
    4. Hou, Li & Wu, Qiang & Xie, Yundong, 2024. "Does open identity of peer reviewers positively relate to citations?," Journal of Informetrics, Elsevier, vol. 18(1).
    5. Sun, Zhuanlan & Clark Cao, C. & Ma, Chao & Li, Yiwei, 2023. "The academic status of reviewers predicts their language use," Journal of Informetrics, Elsevier, vol. 17(4).
    6. Chunli Wei & Jingyi Zhao & Jue Ni & Jiang Li, 2023. "What does open peer review bring to scientific articles? Evidence from PLoS journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(5), pages 2763-2776, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Chunli Wei & Jingyi Zhao & Jue Ni & Jiang Li, 2023. "What does open peer review bring to scientific articles? Evidence from PLoS journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(5), pages 2763-2776, May.
    2. Barbara McGillivray & Mathias Astell, 2019. "The relationship between usage and citations in an open access mega-journal," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(2), pages 817-838, November.
    3. Ying Guo & Xiantao Xiao, 2022. "Author-level altmetrics for the evaluation of Chinese scholars," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(2), pages 973-990, February.
    4. Zhentao Liang & Jin Mao & Kun Lu & Gang Li, 2021. "Finding citations for PubMed: a large-scale comparison between five freely available bibliographic data sources," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(12), pages 9519-9542, December.
    5. Liwei Zhang & Jue Wang, 2021. "What affects publications’ popularity on Twitter?," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(11), pages 9185-9198, November.
    6. Li, Huixu & Liu, Lanjian & Wang, Xianwen, 2021. "The open access effect in social media exposure of scholarly articles: A matched-pair analysis," Journal of Informetrics, Elsevier, vol. 15(3).
    7. Siluo Yang & Xin Xing & Dietmar Wolfram, 2018. "Difference in the impact of open-access papers published by China and the USA," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(2), pages 1017-1037, May.
    8. Guoqiang Liang & Haiyan Hou & Qiao Chen & Zhigang Hu, 2020. "Diffusion and adoption: an explanatory model of “question mark” and “rising star” articles," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(1), pages 219-232, July.
    9. Hajar Sotudeh & Zohreh Estakhr, 2018. "Sustainability of open access citation advantage: the case of Elsevier’s author-pays hybrid open access journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 563-576, April.
    10. Zhiqi Wang & Wolfgang Glänzel & Yue Chen, 2020. "The impact of preprints in Library and Information Science: an analysis of citations, usage and social attention indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1403-1423, November.
    11. Guangyao Zhang & Yuqi Wang & Weixi Xie & Han Du & Chunlin Jiang & Xianwen Wang, 2021. "The open access usage advantage: a temporal and spatial analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 6187-6199, July.
    12. Bikun Chen, 2018. "Usage pattern comparison of the same scholarly articles between Web of Science (WoS) and Springer," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 519-537, April.
    13. Abdelghani Maddi & David Sapinho, 2023. "On the culture of open access: the Sci-hub paradox," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(10), pages 5647-5658, October.
    14. Laura Saraite Sariene & Carmen Caba Pérez & Antonio M López Hernández, 2020. "Expanding the actions of Open Government in higher education sector: From web transparency to Open Science," PLOS ONE, Public Library of Science, vol. 15(9), pages 1-18, September.
    15. Brady D. Lund & Sanjay Kumar Maurya, 2020. "The relationship between highly-cited papers and the frequency of citations to other papers within-issue among three top information science journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2491-2504, December.
    16. W Benedikt Schmal, 2023. "The X Factor: Open Access, New Journals, and Incumbent Competitors," Working Papers of Department of Management, Strategy and Innovation, Leuven 723956, KU Leuven, Faculty of Economics and Business (FEB), Department of Management, Strategy and Innovation, Leuven.
    17. Giovanni Abramo & Ciriaco Andrea D’Angelo & Leonardo Grilli, 2024. "The role of non-scientific factors vis-à-vis the quality of publications in determining their scholarly impact," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(8), pages 5003-5019, August.
    18. Chompunuch Saravudecha & Duangruthai Na Thungfai & Chananthida Phasom & Sodsri Gunta-in & Aorrakanya Metha & Peangkobfah Punyaphet & Tippawan Sookruay & Wannachai Sakuludomkan & Nut Koonrungsesomboon, 2023. "Hybrid Gold Open Access Citation Advantage in Clinical Medicine: Analysis of Hybrid Journals in the Web of Science," Publications, MDPI, vol. 11(2), pages 1-9, March.
    19. Meijun Liu & Dongbo Shi & Jiang Li, 2017. "Double-edged sword of interdisciplinary knowledge flow from hard sciences to humanities and social sciences: Evidence from China," PLOS ONE, Public Library of Science, vol. 12(9), pages 1-16, September.
    20. Kim Holmberg & Han Woo Park, 2018. "An altmetric investigation of the online visibility of South Korea-based scientific journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(1), pages 603-613, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:126:y:2021:i:12:d:10.1007_s11192-021-04182-9. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.