IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v117y2018i3d10.1007_s11192-018-2912-6.html
   My bibliography  Save this article

Are non-monetary rewards effective in attracting peer reviewers? A natural experiment

Author

Listed:
  • Monica Aniela Zaharie

    (Babeş-Bolyai University)

  • Marco Seeber

    (Babeş-Bolyai University
    Ghent University)

Abstract

Editors of scientific journals meet increasing challenges to find peer reviewers. Rewarding reviewers has been proposed as a solution to incentives peer review, and journals have already started to offer different kinds of rewards, particularly non-monetary ones. However, research so far has mainly explored the efficacy of monetary rewards, while research on non-monetary rewards is barely absent. The goal of this article is to fill this gap by exploring whether and under what conditions a rather common non-monetary reward employed by journals, i.e., to recognize reviewers work by publishing their names on a yearly issue, is effective in increasing the willingness of scientists to become peer reviewers. We test the efficacy of three different reward settings identified in the literature: (1) engagement contingent, (2) task-completion contingent, and (3) performance contingent, through a natural experiment involving 1865 scientists in faculties of business and economics of Romanian universities. We explore whether reward efficacy varies across scientists depending on their gender, academic rank, research productivity, and type of institution to which they are affiliated. The results show that the performance contingency strongly reduces the number of respondents willing to become reviewers (− 60 % compared to a no-reward setting), particularly males and research productive scientists. Scientists affiliated with private universities are strongly discouraged by the reward. In sum, the results suggest that non-monetary rewards are not necessarily effective, as in some cases they may actually discourage the most intrinsically motivated and competent reviewers.

Suggested Citation

  • Monica Aniela Zaharie & Marco Seeber, 2018. "Are non-monetary rewards effective in attracting peer reviewers? A natural experiment," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(3), pages 1587-1609, December.
  • Handle: RePEc:spr:scient:v:117:y:2018:i:3:d:10.1007_s11192-018-2912-6
    DOI: 10.1007/s11192-018-2912-6
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-018-2912-6
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s11192-018-2912-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Björk, Bo-Christer & Solomon, David, 2013. "The publishing delay in scholarly peer-reviewed journals," Journal of Informetrics, Elsevier, vol. 7(4), pages 914-923.
    2. Andreas Fuster & Stephan Meier, 2010. "Another Hidden Cost of Incentives: The Detrimental Effect on Norm Enforcement," Management Science, INFORMS, vol. 56(1), pages 57-70, January.
    3. Jean Tirole & Roland Bénabou, 2006. "Incentives and Prosocial Behavior," American Economic Review, American Economic Association, vol. 96(5), pages 1652-1678, December.
    4. Uri Gneezy & Stephan Meier & Pedro Rey-Biel, 2011. "When and Why Incentives (Don't) Work to Modify Behavior," Journal of Economic Perspectives, American Economic Association, vol. 25(4), pages 191-210, Fall.
    5. Richard Van Noorden, 2013. "Company offers portable peer review," Nature, Nature, vol. 494(7436), pages 161-161, February.
    6. Alessandro Bucciol & Natalia Montinari & Marco Piovesan, 2015. "Do Not Trash the Incentive! Monetary Incentives and Waste Sorting," Scandinavian Journal of Economics, Wiley Blackwell, vol. 117(4), pages 1204-1229, October.
    7. José Luis Ortega, 2017. "Are peer-review activities related to reviewer bibliometric performance? A scientometric analysis of Publons," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(2), pages 947-962, August.
    8. Jana Gallus & Bruno S. Frey, 2016. "Awards: A strategic management perspective," Strategic Management Journal, Wiley Blackwell, vol. 37(8), pages 1699-1714, August.
    9. Simone Righi & Károly Takács, 2017. "The miracle of peer review and development in science: an agent-based model," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 587-607, October.
    10. Ofer H. Azar, 2006. "The Academic Review Process: How Can We Make it More Efficient?," The American Economist, Sage Publications, vol. 50(1), pages 37-50, March.
    11. Flaminio Squazzoni & Elise Brezis & Ana Marušić, 2017. "Scientometrics of peer review," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 501-502, October.
    12. Raj Chetty & Emmanuel Saez & Laszlo Sandor, 2014. "What Policies Increase Prosocial Behavior? An Experiment with Referees at the Journal of Public Economics," Journal of Economic Perspectives, American Economic Association, vol. 28(3), pages 169-188, Summer.
    13. Gary D. Thompson & Satheesh V. Aradhyula & George Frisvold & Russell Tronstad, 2010. "Does Paying Referees Expedite Reviews? Results of a Natural Experiment," Southern Economic Journal, John Wiley & Sons, vol. 76(3), pages 678-692, January.
    14. Engers, Maxim & Gans, Joshua S, 1998. "Why Referees Are Not Paid (Enough)," American Economic Review, American Economic Association, vol. 88(5), pages 1341-1349, December.
    15. Carole J. Lee & Cassidy R. Sugimoto & Guo Zhang & Blaise Cronin, 2013. "Bias in peer review," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(1), pages 2-17, January.
    16. Marco Seeber & Alberto Bacchelli, 2017. "Does single blind peer review hinder newcomers?," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 567-585, October.
    17. Tibor Braun & Ildikó Dióspatonyi, 2005. "The journal gatekeepers of major publishing houses of core science journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 64(2), pages 113-120, August.
    18. Heckman, James, 2013. "Sample selection bias as a specification error," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 31(3), pages 129-137.
    19. Squazzoni, Flaminio & Bravo, Giangiacomo & Takács, Károly, 2013. "Does incentive provision increase the quality of peer review? An experimental study," Research Policy, Elsevier, vol. 42(1), pages 287-294.
    20. Dan Ariely & Anat Bracha & Stephan Meier, 2009. "Doing Good or Doing Well? Image Motivation and Monetary Incentives in Behaving Prosocially," American Economic Review, American Economic Association, vol. 99(1), pages 544-555, March.
    21. Daniel S. Hamermesh, 1994. "Facts and Myths about Refereeing," Journal of Economic Perspectives, American Economic Association, vol. 8(1), pages 153-163, Winter.
    22. Juin-jen Chang & Ching-chong Lai, 2001. "Is It Worthwhile to Pay Referees?," Southern Economic Journal, John Wiley & Sons, vol. 68(2), pages 457-463, October.
    23. Pedro Teixeira & Vera Rocha & Ricardo Biscaia & Margarida F. Cardoso, 2014. "Public and private higher education in Europe: competition, complementarity or worlds apart?," Chapters, in: Andrea Bonaccorsi (ed.), Knowledge, Diversity and Performance in European Higher Education, chapter 3, pages iii-iii, Edward Elgar Publishing.
    24. Carole J. Lee & Cassidy R. Sugimoto & Guo Zhang & Blaise Cronin, 2013. "Bias in peer review," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(1), pages 2-17, January.
    25. Azzurra Ragone & Katsiaryna Mirylenka & Fabio Casati & Maurizio Marchese, 2013. "On peer review in computer science: analysis of its effectiveness and suggestions for improvement," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(2), pages 317-356, November.
    26. Jose A. García & Rosa Rodriguez-Sánchez & Joaquín Fdez-Valdivia, 2015. "The principal-agent problem in peer review," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(2), pages 297-308, February.
    27. Sergio Copiello, 2018. "On the money value of peer review," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 613-620, April.
    28. Joshua Angrist & Victor Lavy, 2009. "The Effects of High Stakes High School Achievement Awards: Evidence from a Randomized Trial," American Economic Review, American Economic Association, vol. 99(4), pages 1384-1414, September.
    29. Vivian M Nguyen & Neal R Haddaway & Lee F G Gutowsky & Alexander D M Wilson & Austin J Gallagher & Michael R Donaldson & Neil Hammerschlag & Steven J Cooke, 2015. "How Long Is Too Long in Contemporary Peer Review? Perspectives from Authors Publishing in Conservation Biology Journals," PLOS ONE, Public Library of Science, vol. 10(8), pages 1-20, August.
    30. Zaharie, Monica Aniela & Osoian, Codruţa Luminiţa, 2016. "Peer review motivation frames: A qualitative approach," European Management Journal, Elsevier, vol. 34(1), pages 69-79.
    31. Federico Bianchi & Francisco Grimaldo & Giangiacomo Bravo & Flaminio Squazzoni, 2018. "The peer review game: an agent-based model of scientists facing resource constraints and institutional pressures," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 1401-1420, September.
    32. Gregory B. Northcraft & Ann E. Tenbrunsel, 2011. "Effective Matrices, Decision Frames, and Cooperation in Volunteer Dilemmas: A Theoretical Perspective on Academic Peer Review," Organization Science, INFORMS, vol. 22(5), pages 1277-1285, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Carol Nash, 2023. "Roles and Responsibilities for Peer Reviewers of International Journals," Publications, MDPI, vol. 11(2), pages 1-24, June.
    2. Maria Petrescu & Anjala S. Krishen, 2022. "The evolving crisis of the peer-review process," Journal of Marketing Analytics, Palgrave Macmillan, vol. 10(3), pages 185-186, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sergio Copiello, 2018. "On the money value of peer review," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 613-620, April.
    2. Zaharie, Monica Aniela & Osoian, Codruţa Luminiţa, 2016. "Peer review motivation frames: A qualitative approach," European Management Journal, Elsevier, vol. 34(1), pages 69-79.
    3. Thomas Feliciani & Junwen Luo & Lai Ma & Pablo Lucas & Flaminio Squazzoni & Ana Marušić & Kalpana Shankar, 2019. "A scoping review of simulation models of peer review," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(1), pages 555-594, October.
    4. Canoy Marcel & Veld Daan L. in ’t, 2014. "How to Boost the Production of Free Services: In Search of the Holy Referee Grail," Man and the Economy, De Gruyter, vol. 1(1), pages 1-14, June.
    5. Lokman Tutuncu, 2023. "All-pervading insider bias alters review time in Turkish university journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(6), pages 3743-3791, June.
    6. Zhang, Guangyao & Xu, Shenmeng & Sun, Yao & Jiang, Chunlin & Wang, Xianwen, 2022. "Understanding the peer review endeavor in scientific publishing," Journal of Informetrics, Elsevier, vol. 16(2).
    7. Azar, Ofer H., 2008. "Evolution of social norms with heterogeneous preferences: A general model and an application to the academic review process," Journal of Economic Behavior & Organization, Elsevier, vol. 65(3-4), pages 420-435, March.
    8. David Card & Stefano DellaVigna, 2017. "What do Editors Maximize? Evidence from Four Leading Economics Journals," NBER Working Papers 23282, National Bureau of Economic Research, Inc.
    9. David Card & Stefano DellaVigna, 2020. "What Do Editors Maximize? Evidence from Four Economics Journals," The Review of Economics and Statistics, MIT Press, vol. 102(1), pages 195-217, March.
    10. Maho Nakagawa & Mathieu Lefebvre & Anne Stenger, 2022. "Long-lasting effects of incentives and social preference: A public goods experiment," Post-Print hal-03777681, HAL.
    11. Boyer, Pierre C. & Dwenger, Nadja & Rincke, Johannes, 2016. "Do norms on contribution behavior affect intrinsic motivation? Field-experimental evidence from Germany," Journal of Public Economics, Elsevier, vol. 144(C), pages 140-153.
    12. Tanjim Hossain & King King Li, 2014. "Crowding Out in the Labor Market: A Prosocial Setting Is Necessary," Management Science, INFORMS, vol. 60(5), pages 1148-1160, May.
    13. Takahashi, Hiromasa & Shen, Junyi & Ogawa, Kazuhito, 2016. "An experimental examination of compensation schemes and level of effort in differentiated tasks," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 61(C), pages 12-19.
    14. Squazzoni, Flaminio & Bravo, Giangiacomo & Takács, Károly, 2013. "Does incentive provision increase the quality of peer review? An experimental study," Research Policy, Elsevier, vol. 42(1), pages 287-294.
    15. Marco Seeber & Alberto Bacchelli, 2017. "Does single blind peer review hinder newcomers?," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 567-585, October.
    16. Jing Wang & Gen Li & Kai-Lung Hui, 2022. "Monetary Incentives and Knowledge Spillover: Evidence from a Natural Experiment," Management Science, INFORMS, vol. 68(5), pages 3549-3572, May.
    17. Mortimer, Duncan & Harris, Anthony & Wijnands, Jasper S. & Stevenson, Mark, 2021. "Persistence or reversal? The micro-effects of time-varying financial penalties," Journal of Economic Behavior & Organization, Elsevier, vol. 188(C), pages 72-86.
    18. Christine Exley, 2013. "Incentives for Prosocial Behavior: The Role of Reputations," Discussion Papers 12-022, Stanford Institute for Economic Policy Research.
    19. ITO Koichiro & IDA Takanori & TANAKA Makoto, 2015. "The Persistence of Moral Suasion and Economic Incentives: Field experimental evidence from energy demand," Discussion papers 15014, Research Institute of Economy, Trade and Industry (RIETI).
    20. Ina Ganguli & Marieke Huysentruyt & Chloé Le Coq, 2021. "How Do Nascent Social Entrepreneurs Respond to Rewards? A Field Experiment on Motivations in a Grant Competition," Management Science, INFORMS, vol. 67(10), pages 6294-6316, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:117:y:2018:i:3:d:10.1007_s11192-018-2912-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.