IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v83y2010i2d10.1007_s11192-009-0011-4.html
   My bibliography  Save this article

A content analysis of referees’ comments: how do comments on manuscripts rejected by a high-impact journal and later published in either a low- or high-impact journal differ?

Author

Listed:
  • Lutz Bornmann

    (ETH Zurich)

  • Christophe Weymuth

    (Biosynth AG)

  • Hans-Dieter Daniel

    (ETH Zurich
    University of Zurich)

Abstract

Using the data of a comprehensive evaluation study on the peer review process of Angewandte Chemie International Edition (AC-IE), we examined in this study the way in which referees’ comments differ on manuscripts rejected at AC-IE and later published in either a low-impact journal (Tetrahedron Letters, n = 54) or a high-impact journal (Journal of the American Chemical Society, n = 42). For this purpose, a content analysis was performed of comments which led to the rejection of the manuscripts at AC-IE. For the content analysis, a classification scheme with thematic areas developed by Bornmann et al. (2008) was used. As the results of the analysis demonstrate, a large number of negative comments from referees in the areas “Relevance of contribution” and “Design/Conception” are clear signs that a manuscript rejected at AC-IE will not be published later in a high-impact journal. The number of negative statements in the areas “Writing/Presentation,” “Discussion of results,” “Method/Statistics,” and “Reference to the literature and documentation,” on the other hand, had no statistically significant influence on the probability that a rejected manuscript would later be published in a low- or high-impact journal. The results of this study have various implications for authors, journal editors and referees.

Suggested Citation

  • Lutz Bornmann & Christophe Weymuth & Hans-Dieter Daniel, 2010. "A content analysis of referees’ comments: how do comments on manuscripts rejected by a high-impact journal and later published in either a low- or high-impact journal differ?," Scientometrics, Springer;Akadémiai Kiadó, vol. 83(2), pages 493-506, May.
  • Handle: RePEc:spr:scient:v:83:y:2010:i:2:d:10.1007_s11192-009-0011-4
    DOI: 10.1007/s11192-009-0011-4
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-009-0011-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s11192-009-0011-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Lutz Bornmann & Irina Nast & Hans-Dieter Daniel, 2008. "Do editors and referees look for signs of scientific misconduct when reviewing manuscripts? A quantitative content analysis of studies that examined review criteria and reasons for accepting and rejec," Scientometrics, Springer;Akadémiai Kiadó, vol. 77(3), pages 415-432, December.
    2. J. Scott Long & Jeremy Freese, 2006. "Regression Models for Categorical Dependent Variables using Stata, 2nd Edition," Stata Press books, StataCorp LP, edition 2, number long2, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Meva Bayrak Karsli & Sinem Karabey & Nergiz Ercil Cagiltay & Yuksel Goktas, 2018. "Comparison of the discussion sections of PhD dissertations in educational technology: the case of Turkey and the USA," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(3), pages 1381-1403, December.
    2. Charles W. Fox, 2017. "Difficulty of recruiting reviewers predicts review scores and editorial decisions at six journals of ecology and evolution," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 465-477, October.
    3. Cassidy R. Sugimoto & Blaise Cronin, 2013. "Citation gamesmanship: testing for evidence of ego bias in peer review," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(3), pages 851-862, June.
    4. Niccolò Casnici & Francisco Grimaldo & Nigel Gilbert & Pierpaolo Dondio & Flaminio Squazzoni, 2017. "Assessing peer review by gauging the fate of rejected manuscripts: the case of the Journal of Artificial Societies and Social Simulation," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 533-546, October.
    5. Embiya Celik & Nuray Gedik & Güler Karaman & Turgay Demirel & Yuksel Goktas, 2014. "Mistakes encountered in manuscripts on education and their effects on journal rejections," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(3), pages 1837-1853, March.
    6. Lutz Bornmann & Hermann Schier & Werner Marx & Hans-Dieter Daniel, 2011. "Does the h index for assessing single publications really work? A case study on papers published in chemistry," Scientometrics, Springer;Akadémiai Kiadó, vol. 89(3), pages 835-843, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mikael Svensson & Fredrik Nilsson & Karl Arnberg, 2015. "Reimbursement Decisions for Pharmaceuticals in Sweden: The Impact of Disease Severity and Cost Effectiveness," PharmacoEconomics, Springer, vol. 33(11), pages 1229-1236, November.
    2. Erik Stam & Roy Thurik & Peter van der Zwan, 2010. "Entrepreneurial exit in real and imagined markets," Industrial and Corporate Change, Oxford University Press, vol. 19(4), pages 1109-1139, August.
    3. Miriam Marcén & Marina Morales, 2019. "Live together: does culture matter?," Review of Economics of the Household, Springer, vol. 17(2), pages 671-713, June.
    4. Krishna Chaitanya Vadlamannati & Yuanxin Li & Samuel Brazys & Alexander Dukalskis, 2019. "Building Bridges or Breaking Bonds? The Belt and Road Initiative and Foreign Aid Competition," Working Papers 201906, Geary Institute, University College Dublin.
    5. Altorjai, Szilvia, 2013. "Over-qualification of immigrants in the UK," ISER Working Paper Series 2013-11, Institute for Social and Economic Research.
    6. Michelsen, Carl Christian & Madlener, Reinhard, 2016. "Switching from fossil fuel to renewables in residential heating systems: An empirical study of homeowners' decisions in Germany," Energy Policy, Elsevier, vol. 89(C), pages 95-105.
    7. Susanne Meyer & Javier Revilla Diez, 2015. "One country, two systems: How regional institutions shape governance modes in the greater Pearl River Delta, China," Papers in Regional Science, Wiley Blackwell, vol. 94(4), pages 891-900, November.
    8. Fabrizio Pompei & Ekaterina Selezneva, 2015. "Education Mismatch, Human Capital and Labour Status of Young People across European Union Countries," Working Papers 347, Leibniz Institut für Ost- und Südosteuropaforschung (Institute for East and Southeast European Studies).
    9. Lisa Cook, 2014. "Violence and economic activity: evidence from African American patents, 1870–1940," Journal of Economic Growth, Springer, vol. 19(2), pages 221-257, June.
    10. Bornmann, Lutz & Leydesdorff, Loet & Wang, Jian, 2014. "How to improve the prediction based on citation impact percentiles for years shortly after the publication date?," Journal of Informetrics, Elsevier, vol. 8(1), pages 175-180.
    11. Battke, Benedikt & Schmidt, Tobias S. & Stollenwerk, Stephan & Hoffmann, Volker H., 2016. "Internal or external spillovers—Which kind of knowledge is more likely to flow within or across technologies," Research Policy, Elsevier, vol. 45(1), pages 27-41.
    12. Niklas Elert, 2014. "What determines entry? Evidence from Sweden," The Annals of Regional Science, Springer;Western Regional Science Association, vol. 53(1), pages 55-92, August.
    13. Kelly Sharp & Hisham Zerriffi & Philippe Billon, 2020. "Land scarcity, resettlement and food security: Assessing the effect of voluntary resettlement on diet quality in Malawi," Food Security: The Science, Sociology and Economics of Food Production and Access to Food, Springer;The International Society for Plant Pathology, vol. 12(1), pages 191-205, February.
    14. Ryan J. Owens, 2010. "The Separation of Powers and Supreme Court Agenda Setting," American Journal of Political Science, John Wiley & Sons, vol. 54(2), pages 412-427, April.
    15. Lee, Richard J. & Sener, Ipek N. & Mokhtarian, Patricia L. & Handy, Susan L., 2017. "Relationships between the online and in-store shopping frequency of Davis, California residents," Transportation Research Part A: Policy and Practice, Elsevier, vol. 100(C), pages 40-52.
    16. Youssef Zizi & Mohamed Oudgou & Abdeslam El Moudden, 2020. "Determinants and Predictors of SMEs’ Financial Failure: A Logistic Regression Approach," Risks, MDPI, Open Access Journal, vol. 8(4), pages 1-21, October.
    17. Caudill, Jonathan W. & Getty, Ryan & Smith, Rick & Patten, Ryan & Trulson, Chad R., 2013. "Discouraging window breakers: The lagged effects of police activity on crime," Journal of Criminal Justice, Elsevier, vol. 41(1), pages 18-23.
    18. Martina Neuländtner & Thomas Scherngell, 0. "Geographical or relational: What drives technology-specific R&D collaboration networks?," The Annals of Regional Science, Springer;Western Regional Science Association, vol. 0, pages 1-31.
    19. Stoeffler, Quentin & Mills, Bradford & del Ninno, Carlo, 2016. "Reaching the Poor: Cash Transfer Program Targeting in Cameroon," World Development, Elsevier, vol. 83(C), pages 244-263.
    20. Zhou, Min & Guo, Wei, 2021. "Social factors and worry associated with COVID-19: Evidence from a large survey in China," Social Science & Medicine, Elsevier, vol. 277(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:83:y:2010:i:2:d:10.1007_s11192-009-0011-4. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://www.springer.com .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.