IDEAS home Printed from https://ideas.repec.org/a/eee/respol/v42y2013i1p287-294.html
   My bibliography  Save this article

Does incentive provision increase the quality of peer review? An experimental study

Author

Listed:
  • Squazzoni, Flaminio
  • Bravo, Giangiacomo
  • Takács, Károly

Abstract

Although peer review is crucial for innovation and experimental discoveries in science, it is poorly understood in scientific terms. Discovering its true dynamics and exploring adjustments which improve the commitment of everyone involved could benefit scientific development for all disciplines and consequently increase innovation in the economy and the society. We have reported the results of an innovative experiment developed to model peer review. We demonstrate that offering material rewards to referees tends to decrease the quality and efficiency of the reviewing process. Our findings help to discuss the viability of different options of incentive provision, supporting the idea that journal editors and responsible of research funding agencies should be extremely careful in offering material incentives on reviewing, since these might undermine moral motives which guide referees’ behavior.

Suggested Citation

  • Squazzoni, Flaminio & Bravo, Giangiacomo & Takács, Károly, 2013. "Does incentive provision increase the quality of peer review? An experimental study," Research Policy, Elsevier, vol. 42(1), pages 287-294.
  • Handle: RePEc:eee:respol:v:42:y:2013:i:1:p:287-294
    DOI: 10.1016/j.respol.2012.04.014
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0048733312001230
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.respol.2012.04.014?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ernst Fehr & Klaus M. Schmidt, 1999. "A Theory of Fairness, Competition, and Cooperation," The Quarterly Journal of Economics, Oxford University Press, vol. 114(3), pages 817-868.
    2. Flaminio Squazzoni & Károly Takács, 2011. "Social Simulation That 'Peers into Peer Review'," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 14(4), pages 1-3.
    3. David N. Laband, 1990. "Is There Value-Added from the Review Process in Economics?: Preliminary Evidence from Authors," The Quarterly Journal of Economics, Oxford University Press, vol. 105(2), pages 341-352.
    4. Daniel S. Hamermesh, 1994. "Facts and Myths about Refereeing," Journal of Economic Perspectives, American Economic Association, vol. 8(1), pages 153-163, Winter.
    5. Glenn Ellison, 2002. "The Slowdown of the Economics Publishing Process," Journal of Political Economy, University of Chicago Press, vol. 110(5), pages 947-993, October.
    6. Azar, Ofer H., 2008. "Evolution of social norms with heterogeneous preferences: A general model and an application to the academic review process," Journal of Economic Behavior & Organization, Elsevier, vol. 65(3-4), pages 420-435, March.
    7. Juin-jen Chang & Ching-chong Lai, 2001. "Is It Worthwhile to Pay Referees?," Southern Economic Journal, John Wiley & Sons, vol. 68(2), pages 457-463, October.
    8. Almås, Ingvild & Cappelen, Alexander W. & Sørensen, Erik Ø. & Tungodden, Bertil, 2015. "Fairness and the Development of Inequality Acceptance," Discussion Paper Series in Economics 18/2015, Norwegian School of Economics, Department of Economics.
    9. Herbert Gintis, 2000. "Strong Reciprocity and Human Sociality," UMASS Amherst Economics Working Papers 2000-02, University of Massachusetts Amherst, Department of Economics.
    10. Kenneth Arrow, 1962. "Economic Welfare and the Allocation of Resources for Invention," NBER Chapters, in: The Rate and Direction of Inventive Activity: Economic and Social Factors, pages 609-626, National Bureau of Economic Research, Inc.
    11. Scott Stern, 2004. "Do Scientists Pay to Be Scientists?," Management Science, INFORMS, vol. 50(6), pages 835-853, June.
    12. Partha, Dasgupta & David, Paul A., 1994. "Toward a new economics of science," Research Policy, Elsevier, vol. 23(5), pages 487-521, September.
    13. Bruno S. Frey & Reto Jegen, 2001. "Motivation Crowding Theory," Journal of Economic Surveys, Wiley Blackwell, vol. 15(5), pages 589-611, December.
    14. Greiner, Ben, 2004. "An Online Recruitment System for Economic Experiments," MPRA Paper 13513, University Library of Munich, Germany.
    15. Johnson, Noel D. & Mislin, Alexandra A., 2011. "Trust games: A meta-analysis," Journal of Economic Psychology, Elsevier, vol. 32(5), pages 865-889.
    16. Haeussler, Carolin, 2011. "Information-sharing in academia and the industry: A comparative study," Research Policy, Elsevier, vol. 40(1), pages 105-122, February.
    17. Andreas Ortmann & John Fitzgerald & Carl Boeing, 2000. "Trust, Reciprocity, and Social History: A Re-examination," Experimental Economics, Springer;Economic Science Association, vol. 3(1), pages 81-100, June.
    18. Stephan, Paula E., 2010. "The Economics of Science," Handbook of the Economics of Innovation, in: Bronwyn H. Hall & Nathan Rosenberg (ed.), Handbook of the Economics of Innovation, edition 1, volume 1, chapter 0, pages 217-273, Elsevier.
    19. Engers, Maxim & Gans, Joshua S, 1998. "Why Referees Are Not Paid (Enough)," American Economic Review, American Economic Association, vol. 88(5), pages 1341-1349, December.
    20. Hendrik P. van Dalen & K?ne Henkens, 2005. "Signals in science - On the importance of signaling in gaining attention in science," Scientometrics, Springer;Akadémiai Kiadó, vol. 64(2), pages 209-233, August.
    21. Rabin, Matthew, 1993. "Incorporating Fairness into Game Theory and Economics," American Economic Review, American Economic Association, vol. 83(5), pages 1281-1302, December.
    22. Pollak, Robert A & Wachter, Michael L, 1975. "The Relevance of the Household Production Function and Its Implications for the Allocation of Time," Journal of Political Economy, University of Chicago Press, vol. 83(2), pages 255-277, April.
    23. Paul A. David, 2004. "Can 'Open Science' be Protected from the Evolving Scheme of IPR Protections?," Journal of Institutional and Theoretical Economics (JITE), Mohr Siebeck, Tübingen, vol. 160(1), pages 9-34, March.
    24. Ofer H. Azar, 2006. "The Academic Review Process: How Can We Make it More Efficient?," The American Economist, Sage Publications, vol. 50(1), pages 37-50, March.
    25. Roach, Michael & Sauermann, Henry, 2010. "A taste for science? PhD scientists' academic orientation and self-selection into research careers in industry," Research Policy, Elsevier, vol. 39(3), pages 422-434, April.
    26. Berg Joyce & Dickhaut John & McCabe Kevin, 1995. "Trust, Reciprocity, and Social History," Games and Economic Behavior, Elsevier, vol. 10(1), pages 122-142, July.
    27. Richard R. Nelson, 1959. "The Simple Economics of Basic Scientific Research," Journal of Political Economy, University of Chicago Press, vol. 67, pages 297-297.
    28. Urs Fischbacher, 2007. "z-Tree: Zurich toolbox for ready-made economic experiments," Experimental Economics, Springer;Economic Science Association, vol. 10(2), pages 171-178, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Vladimir Batagelj & Anuška Ferligoj & Flaminio Squazzoni, 2017. "The emergence of a field: a network analysis of research on peer review," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 503-532, October.
    2. Simone Righi & Károly Takács, 2017. "The miracle of peer review and development in science: an agent-based model," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 587-607, October.
    3. Raj Chetty & Emmanuel Saez & Laszlo Sandor, 2014. "What Policies Increase Prosocial Behavior? An Experiment with Referees at the Journal of Public Economics," Journal of Economic Perspectives, American Economic Association, vol. 28(3), pages 169-188, Summer.
    4. Richard R Snell, 2015. "Menage a Quoi? Optimal Number of Peer Reviewers," PLOS ONE, Public Library of Science, vol. 10(4), pages 1-14, April.
    5. Takahashi, Hiromasa & Shen, Junyi & Ogawa, Kazuhito, 2016. "An experimental examination of compensation schemes and level of effort in differentiated tasks," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 61(C), pages 12-19.
    6. Vivian M Nguyen & Neal R Haddaway & Lee F G Gutowsky & Alexander D M Wilson & Austin J Gallagher & Michael R Donaldson & Neil Hammerschlag & Steven J Cooke, 2015. "How Long Is Too Long in Contemporary Peer Review? Perspectives from Authors Publishing in Conservation Biology Journals," PLOS ONE, Public Library of Science, vol. 10(8), pages 1-20, August.
    7. Mario Paolucci & Francisco Grimaldo, 2014. "Mechanism change in a simulation of peer review: from junk support to elitism," Scientometrics, Springer;Akadémiai Kiadó, vol. 99(3), pages 663-688, June.
    8. Marco Seeber & Alberto Bacchelli, 2017. "Does single blind peer review hinder newcomers?," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 567-585, October.
    9. Olgica Nedić & Aleksandar Dekanski, 2016. "Priority criteria in peer review of scientific articles," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(1), pages 15-26, April.
    10. Simone Righi & Karoly Takacs, 2016. "The Miracle of Peer Review and Development inScience: An Agent-Based Model," Department of Economics 0090, University of Modena and Reggio E., Faculty of Economics "Marco Biagi".
    11. Christian Matt & Christian Hoerndlein & Thomas Hess, 2017. "Let the crowd be my peers? How researchers assess the prospects of social peer review," Electronic Markets, Springer;IIM University of St. Gallen, vol. 27(2), pages 111-124, May.
    12. José Luis Ortega, 2017. "Are peer-review activities related to reviewer bibliometric performance? A scientometric analysis of Publons," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(2), pages 947-962, August.
    13. Christian Matt & Christian Hoerndlein & Thomas Hess, 0. "Let the crowd be my peers? How researchers assess the prospects of social peer review," Electronic Markets, Springer;IIM University of St. Gallen, vol. 0, pages 1-14.
    14. Federico Bianchi & Francisco Grimaldo & Giangiacomo Bravo & Flaminio Squazzoni, 2018. "The peer review game: an agent-based model of scientists facing resource constraints and institutional pressures," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 1401-1420, September.
    15. Sergio Copiello, 2018. "On the money value of peer review," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 613-620, April.
    16. Bianchi, Federico & Grimaldo, Francisco & Squazzoni, Flaminio, 2019. "The F3-index. Valuing reviewers for scholarly journals," Journal of Informetrics, Elsevier, vol. 13(1), pages 78-86.
    17. Ayoubi, Charles & Thurm, Boris, 2020. "Knowledge Diffusion and Morality: Why do we Freely Share Valuable Information with Strangers?," OSF Preprints 78mua, Center for Open Science.
    18. Siddarth Srinivasan & Jamie Morgenstern, 2021. "Auctions and Prediction Markets for Scientific Peer Review," Papers 2109.00923, arXiv.org.
    19. Monica Aniela Zaharie & Marco Seeber, 2018. "Are non-monetary rewards effective in attracting peer reviewers? A natural experiment," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(3), pages 1587-1609, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Simeth, Markus & Raffo, Julio D., 2013. "What makes companies pursue an Open Science strategy?," Research Policy, Elsevier, vol. 42(9), pages 1531-1543.
    2. Ofer H. Azar, 2006. "The Academic Review Process: How Can We Make it More Efficient?," The American Economist, Sage Publications, vol. 50(1), pages 37-50, March.
    3. Azar, Ofer H., 2008. "Evolution of social norms with heterogeneous preferences: A general model and an application to the academic review process," Journal of Economic Behavior & Organization, Elsevier, vol. 65(3-4), pages 420-435, March.
    4. Ganglmair, Bernhard & Holcomb, Alex & Myung, Noah, 2020. "Expectations of reciprocity when competitors share information: Experimental evidence," Journal of Economic Behavior & Organization, Elsevier, vol. 170(C), pages 244-267.
    5. Güth, Werner & Mugera, Harriet & Musau, Andrew & Ploner, Matteo, 2014. "Deterministic versus probabilistic consequences of trust and trustworthiness: An experimental investigation," Journal of Economic Psychology, Elsevier, vol. 42(C), pages 28-40.
    6. Simeth, Markus & Lhuillery, Stephane, 2015. "How do firms develop capabilities for scientific disclosure?," Research Policy, Elsevier, vol. 44(7), pages 1283-1295.
    7. Stephan, Paula E., 2010. "The Economics of Science," Handbook of the Economics of Innovation, in: Bronwyn H. Hall & Nathan Rosenberg (ed.), Handbook of the Economics of Innovation, edition 1, volume 1, chapter 0, pages 217-273, Elsevier.
    8. Calabuig, Vicente & Fatas, Enrique & Olcina, Gonzalo & Rodriguez-Lara, Ismael, 2016. "Carry a big stick, or no stick at all," Journal of Economic Psychology, Elsevier, vol. 57(C), pages 153-171.
    9. Daniel Woods & Maroš Servátka, 2019. "Nice to you, nicer to me: Does self-serving generosity diminish the reciprocal response?," Experimental Economics, Springer;Economic Science Association, vol. 22(2), pages 506-529, June.
    10. Andreoli-Versbach, Patrick & Mueller-Langer, Frank, 2014. "Open access to data: An ideal professed but not practised," Research Policy, Elsevier, vol. 43(9), pages 1621-1633.
    11. Gans, Joshua S. & Murray, Fiona E. & Stern, Scott, 2017. "Contracting over the disclosure of scientific knowledge: Intellectual property and academic publication," Research Policy, Elsevier, vol. 46(4), pages 820-835.
    12. Nikos Nikiforakis & Helen Mitchell, 2014. "Mixing the carrots with the sticks: third party punishment and reward," Experimental Economics, Springer;Economic Science Association, vol. 17(1), pages 1-23, March.
    13. Bartoš, Vojtěch & Levely, Ian, 2021. "Sanctioning and trustworthiness across ethnic groups: Experimental evidence from Afghanistan," Journal of Public Economics, Elsevier, vol. 194(C).
    14. Jeongbin Kim & Louis Putterman & Xinyi Zhang, 2019. ""Trust, Beliefs and Cooperation: Excavating a Foundation of Strong Economics," Working Papers 2019-10, Brown University, Department of Economics.
    15. Luigino Bruni & Fabio Tufano, 2017. "The value of vulnerability: The transformative capacity of risky trust," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 12(4), pages 408-414, July.
    16. Lacetera, Nicola & Zirulia, Lorenzo, 2012. "Individual preferences, organization, and competition in a model of R&D incentive provision," Journal of Economic Behavior & Organization, Elsevier, vol. 84(2), pages 550-570.
    17. Margit Osterloh & Bruno S. Frey, 2010. "Academic rankings and research governance," IEW - Working Papers 482, Institute for Empirical Research in Economics - University of Zurich.
    18. Giovanni Di Bartolomeo & Stefano Papa, 2019. "The Effects of Physical Activity on Social Interactions: The Case of Trust and Trustworthiness," Journal of Sports Economics, , vol. 20(1), pages 50-71, January.
    19. Margit Osterloh & Bruno S. Frey, 2009. "Research Governance in Academia: Are there Alternatives to Academic Rankings?," CESifo Working Paper Series 2797, CESifo.
    20. Kurt A. Ackermann & Jürgen Fleiß & Ryan O. Murphy, 2013. "Reciprocity as an individual difference," Working Paper Series, Social and Economic Sciences 2013-05, Faculty of Social and Economic Sciences, Karl-Franzens-University Graz.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:respol:v:42:y:2013:i:1:p:287-294. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://www.elsevier.com/locate/respol .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/respol .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.