IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v91y2012i3d10.1007_s11192-011-0547-y.html
   My bibliography  Save this article

The Hawthorne effect in journal peer review

Author

Listed:
  • Lutz Bornmann

    (Max Planck Society, Administrative Headquarters)

Abstract

Purpose—this paper aims to look at the Hawthorne effect in editorial peer review. Design/methodology/approach—discusses the quality evaluation of refereed scholarly journals. Findings—a key finding of this research was that in the peer review process of one and the same manuscript, reviewers or editors, respectively, arrive at different judgments. This phenomenon is named as “Hawthorne effect” because the different judgements are dependent on the specific conditions under which the peer review process at the individual journals takes place. Originality/value—provides a discussion on the quality evaluation of scholarly journals.

Suggested Citation

  • Lutz Bornmann, 2012. "The Hawthorne effect in journal peer review," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(3), pages 857-862, June.
    • Handle: RePEc:spr:scient:v:91:y:2012:i:3:d:10.1007_s11192-011-0547-y
    DOI: 10.1007/s11192-011-0547-y
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-011-0547-y
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-011-0547-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Joshua S. Gans & George B. Shepherd, 1994. "How Are the Mighty Fallen: Rejected Classic Articles by Leading Economists," Journal of Economic Perspectives, American Economic Association, vol. 8(1), pages 165-179, Winter.
    2. Benda, Wim G.G. & Engels, Tim C.E., 2011. "The predictive validity of peer review: A selective review of the judgmental forecasting qualities of peers, and implications for innovation in science," International Journal of Forecasting, Elsevier, vol. 27(1), pages 166-182.
    3. Bernd Pulverer, 2010. "Transparency showcases strength of peer review," Nature, Nature, vol. 468(7320), pages 29-31, November.
    4. Lipworth, Wendy L. & Kerridge, Ian H. & Carter, Stacy M. & Little, Miles, 2011. "Journal peer review in context: A qualitative study of the social and subjective dimensions of manuscript review in biomedical publishing," Social Science & Medicine, Elsevier, vol. 72(7), pages 1056-1063, April.
    5. Benda, Wim G.G. & Engels, Tim C.E., 2011. "The predictive validity of peer review: A selective review of the judgmental forecasting qualities of peers, and implications for innovation in science," International Journal of Forecasting, Elsevier, vol. 27(1), pages 166-182, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Rosa Rodriguez-Sánchez & J. A. García & J. Fdez-Valdivia, 2018. "Editorial decisions with informed and uninformed reviewers," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(1), pages 25-43, October.
    2. Bornmann, Lutz & Williams, Richard, 2017. "Can the journal impact factor be used as a criterion for the selection of junior researchers? A large-scale empirical study based on ResearcherID data," Journal of Informetrics, Elsevier, vol. 11(3), pages 788-799.
    3. Cortés-Sánchez, Julián David, 2019. "Bibliometric outlook of the most cited documents in business, management and accounting in Ibero-America," SocArXiv cqa3z, Center for Open Science.
    4. Emre Özel, 2024. "What is Gender Bias in Grant Peer review?," Working Papers halshs-03862027, HAL.
    5. O. Mryglod & R. Kenna & Yu. Holovatch & B. Berche, 2013. "Comparison of a citation-based indicator and peer review for absolute and specific measures of research-group excellence," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(3), pages 767-777, December.
    6. J. A. García & Rosa Rodriguez-Sánchez & J. Fdez-Valdivia, 2015. "The author–editor game," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(1), pages 361-380, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Richard R Snell, 2015. "Menage a Quoi? Optimal Number of Peer Reviewers," PLOS ONE, Public Library of Science, vol. 10(4), pages 1-14, April.
    2. van den Besselaar, Peter & Sandström, Ulf, 2015. "Early career grants, performance, and careers: A study on predictive validity of grant decisions," Journal of Informetrics, Elsevier, vol. 9(4), pages 826-838.
    3. Frederik T. Verleysen & Tim C.E. Engels, 2013. "A label for peer-reviewed books," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(2), pages 428-430, February.
    4. Krist Vaesen & Joel Katzav, 2017. "How much would each researcher receive if competitive government research funding were distributed equally among researchers?," PLOS ONE, Public Library of Science, vol. 12(9), pages 1-11, September.
    5. Girish Mallapragada & Nandini Lahiri & Atul Nerkar, 2016. "Peer Review and Research Impact," Customer Needs and Solutions, Springer;Institute for Sustainable Innovation and Growth (iSIG), vol. 3(1), pages 29-41, March.
    6. Lutz Bornmann, 2015. "Interrater reliability and convergent validity of F1000Prime peer review," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(12), pages 2415-2426, December.
    7. Wright, George & Rowe, Gene, 2011. "Group-based judgmental forecasting: An integration of extant knowledge and the development of priorities for a new research agenda," International Journal of Forecasting, Elsevier, vol. 27(1), pages 1-13, January.
    8. Jens Jirschitzka & Aileen Oeberst & Richard Göllner & Ulrike Cress, 2017. "Inter-rater reliability and validity of peer reviews in an interdisciplinary field," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 1059-1092, November.
    9. repec:eee:intfor:v:27:y:2011:i:1:p:1-13 is not listed on IDEAS
    10. Gill, Chelsea & Mehrotra, Vishal & Moses, Olayinka & Bui, Binh, 2023. "The impact of the pitching research framework on AFAANZ grant applications," Pacific-Basin Finance Journal, Elsevier, vol. 77(C).
    11. Girish Mallapragada & Nandini Lahiri & Atul Nerkar, 2016. "Peer Review and Research Impact," Customer Needs and Solutions, Springer;Institute for Sustainable Innovation and Growth (iSIG), vol. 3(1), pages 29-41, March.
    12. Marion Gaspard & Antoine Missemer, 2019. "An inquiry into the Ramsey-Hotelling connection," The European Journal of the History of Economic Thought, Taylor & Francis Journals, vol. 26(2), pages 352-379, March.
    13. Marco LiCalzi, 2022. "Bipartite choices," Decisions in Economics and Finance, Springer;Associazione per la Matematica, vol. 45(2), pages 551-568, December.
    14. Medoff, Marshall H., 2003. "Collaboration and the quality of economics research," Labour Economics, Elsevier, vol. 10(5), pages 597-608, October.
    15. Rosser, J. Jr., 1996. "Development, geography, and economic theory : Paul Krugman (The MIT Press, Cambridge, MA, 1995) pp. iii + 117, index, $20.00," Journal of Economic Behavior & Organization, Elsevier, vol. 31(3), pages 450-454, December.
    16. David B. Audretsch & Christina Guenther & Adam Lederer, 2022. "Publishing in Small Business Economics: An Entrepreneurship Journal," Small Business Economics, Springer, vol. 58(1), pages 1-5, January.
    17. Brogaard, Jonathan & Engelberg, Joseph & Parsons, Christopher A., 2014. "Networks and productivity: Causal evidence from editor rotations," Journal of Financial Economics, Elsevier, vol. 111(1), pages 251-270.
    18. Hendrik P. van Dalen, 1999. "The Golden Age of Nobel Economists," The American Economist, Sage Publications, vol. 43(2), pages 19-35, October.
    19. Hendrik P. van Dalen, 2003. "Pluralism in Economics: A Public Good or a Public Bad?," Tinbergen Institute Discussion Papers 03-034/1, Tinbergen Institute, revised 18 May 2004.
    20. Bruno Frey, 2005. "Problems with Publishing: Existing State and Solutions," European Journal of Law and Economics, Springer, vol. 19(2), pages 173-190, April.
    21. Dell'Anno, Roberto & Caferra, Rocco & Morone, Andrea, 2020. "A “Trojan Horse” in the peer-review process of fee-charging economic journals," Journal of Informetrics, Elsevier, vol. 14(3).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:91:y:2012:i:3:d:10.1007_s11192-011-0547-y. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.