IDEAS home Printed from
   My bibliography  Save this article

Does incentive provision increase the quality of peer review? An experimental study


  • Squazzoni, Flaminio
  • Bravo, Giangiacomo
  • Takács, Károly


Although peer review is crucial for innovation and experimental discoveries in science, it is poorly understood in scientific terms. Discovering its true dynamics and exploring adjustments which improve the commitment of everyone involved could benefit scientific development for all disciplines and consequently increase innovation in the economy and the society. We have reported the results of an innovative experiment developed to model peer review. We demonstrate that offering material rewards to referees tends to decrease the quality and efficiency of the reviewing process. Our findings help to discuss the viability of different options of incentive provision, supporting the idea that journal editors and responsible of research funding agencies should be extremely careful in offering material incentives on reviewing, since these might undermine moral motives which guide referees’ behavior.

Suggested Citation

  • Squazzoni, Flaminio & Bravo, Giangiacomo & Takács, Károly, 2013. "Does incentive provision increase the quality of peer review? An experimental study," Research Policy, Elsevier, vol. 42(1), pages 287-294.
  • Handle: RePEc:eee:respol:v:42:y:2013:i:1:p:287-294 DOI: 10.1016/j.respol.2012.04.014

    Download full text from publisher

    File URL:
    Download Restriction: Full text for ScienceDirect subscribers only

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    1. Ernst Fehr & Klaus M. Schmidt, 1999. "A Theory of Fairness, Competition, and Cooperation," The Quarterly Journal of Economics, Oxford University Press, vol. 114(3), pages 817-868.
    2. Daniel S. Hamermesh, 1994. "Facts and Myths about Refereeing," Journal of Economic Perspectives, American Economic Association, vol. 8(1), pages 153-163, Winter.
    3. Riccardo Boero & Giangiacomo Bravo & Marco Castellani & Francesco Laganà & Flaminio Squazzoni, 2009. "Pillars of Trust: An Experimental Study on Reputation and Its Effects," Sociological Research Online, Sociological Research Online, vol. 14(5), pages 1-5.
    4. Herbert Gintis, 2000. "Strong Reciprocity and Human Sociality," UMASS Amherst Economics Working Papers 2000-02, University of Massachusetts Amherst, Department of Economics.
    5. Kenneth Arrow, 1962. "Economic Welfare and the Allocation of Resources for Invention," NBER Chapters,in: The Rate and Direction of Inventive Activity: Economic and Social Factors, pages 609-626 National Bureau of Economic Research, Inc.
    6. Johnson, Noel D. & Mislin, Alexandra A., 2011. "Trust games: A meta-analysis," Journal of Economic Psychology, Elsevier, vol. 32(5), pages 865-889.
    7. Stephan, Paula E., 2010. "The Economics of Science," Handbook of the Economics of Innovation, Elsevier.
    8. repec:spr:scient:v:64:y:2005:i:2:d:10.1007_s11192-005-0248-5 is not listed on IDEAS
    9. Ofer H. Azar, 2006. "The Academic Review Process: How Can We Make it More Efficient?," The American Economist, Sage Publications, vol. 50(1), pages 37-50, March.
    10. Roach, Michael & Sauermann, Henry, 2010. "A taste for science? PhD scientists' academic orientation and self-selection into research careers in industry," Research Policy, Elsevier, vol. 39(3), pages 422-434, April.
    11. Haeussler, Carolin, 2011. "Information-sharing in academia and the industry: A comparative study," Research Policy, Elsevier, vol. 40(1), pages 105-122, February.
    12. Richard R. Nelson, 1959. "The Simple Economics of Basic Scientific Research," Journal of Political Economy, University of Chicago Press, vol. 67, pages 297-297.
    13. Urs Fischbacher, 2007. "z-Tree: Zurich toolbox for ready-made economic experiments," Experimental Economics, Springer;Economic Science Association, vol. 10(2), pages 171-178, June.
    14. Flaminio Squazzoni & Károly Takács, 2011. "Social Simulation That 'Peers into Peer Review'," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 14(4), pages 1-3.
    15. Juin-jen Chang & Ching-chong Lai, 2001. "Is It Worthwhile to Pay Referees?," Southern Economic Journal, Southern Economic Association, vol. 68(2), pages 457-463, October.
    16. David N. Laband, 1990. "Is There Value-Added from the Review Process in Economics?: Preliminary Evidence from Authors," The Quarterly Journal of Economics, Oxford University Press, vol. 105(2), pages 341-352.
    17. Glenn Ellison, 2002. "The Slowdown of the Economics Publishing Process," Journal of Political Economy, University of Chicago Press, vol. 110(5), pages 947-993, October.
    18. Azar, Ofer H., 2008. "Evolution of social norms with heterogeneous preferences: A general model and an application to the academic review process," Journal of Economic Behavior & Organization, Elsevier, vol. 65(3-4), pages 420-435, March.
    19. Almås, Ingvild & Cappelen, Alexander W. & Sørensen, Erik Ø. & Tungodden, Bertil, 2015. "Fairness and the Development of Inequality Acceptance," Discussion Paper Series in Economics 18/2015, Norwegian School of Economics, Department of Economics.
    20. Scott Stern, 2004. "Do Scientists Pay to Be Scientists?," Management Science, INFORMS, vol. 50(6), pages 835-853, June.
    21. Partha, Dasgupta & David, Paul A., 1994. "Toward a new economics of science," Research Policy, Elsevier, vol. 23(5), pages 487-521, September.
    22. Frey, Bruno S & Jegen, Reto, 2001. " Motivation Crowding Theory," Journal of Economic Surveys, Wiley Blackwell, vol. 15(5), pages 589-611, December.
    23. Greiner, Ben, 2004. "An Online Recruitment System for Economic Experiments," MPRA Paper 13513, University Library of Munich, Germany.
    24. Andreas Ortmann & John Fitzgerald & Carl Boeing, 2000. "Trust, Reciprocity, and Social History: A Re-examination," Experimental Economics, Springer;Economic Science Association, vol. 3(1), pages 81-100, June.
    25. Engers, Maxim & Gans, Joshua S, 1998. "Why Referees Are Not Paid (Enough)," American Economic Review, American Economic Association, vol. 88(5), pages 1341-1349, December.
    26. Hendrik P. van Dalen & K?ne Henkens, 2005. "Signals in science - On the importance of signaling in gaining attention in science," Scientometrics, Springer;Akadémiai Kiadó, vol. 64(2), pages 209-233, August.
    27. Rabin, Matthew, 1993. "Incorporating Fairness into Game Theory and Economics," American Economic Review, American Economic Association, vol. 83(5), pages 1281-1302, December.
    28. Pollak, Robert A & Wachter, Michael L, 1975. "The Relevance of the Household Production Function and Its Implications for the Allocation of Time," Journal of Political Economy, University of Chicago Press, vol. 83(2), pages 255-277, April.
    29. Paul A. David, 2004. "Can 'Open Science' be Protected from the Evolving Scheme of IPR Protections?," Journal of Institutional and Theoretical Economics (JITE), Mohr Siebeck, Tübingen, vol. 160(1), pages 1-9, March.
    30. Berg Joyce & Dickhaut John & McCabe Kevin, 1995. "Trust, Reciprocity, and Social History," Games and Economic Behavior, Elsevier, vol. 10(1), pages 122-142, July.
    Full references (including those not matched with items on IDEAS)


    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.

    Cited by:

    1. Christian Matt & Christian Hoerndlein & Thomas Hess, 0. "Let the crowd be my peers? How researchers assess the prospects of social peer review," Electronic Markets, Springer;IIM University of St. Gallen, vol. 0, pages 1-14.
    2. repec:spr:scient:v:113:y:2017:i:1:d:10.1007_s11192-017-2522-8 is not listed on IDEAS
    3. repec:spr:scient:v:113:y:2017:i:1:d:10.1007_s11192-017-2244-y is not listed on IDEAS
    4. Takahashi, Hiromasa & Shen, Junyi & Ogawa, Kazuhito, 2016. "An experimental examination of compensation schemes and level of effort in differentiated tasks," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 61(C), pages 12-19.
    5. repec:spr:scient:v:99:y:2014:i:3:d:10.1007_s11192-014-1239-1 is not listed on IDEAS
    6. repec:spr:scient:v:113:y:2017:i:1:d:10.1007_s11192-017-2264-7 is not listed on IDEAS
    7. Olgica Nedić & Aleksandar Dekanski, 2016. "Priority criteria in peer review of scientific articles," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(1), pages 15-26, April.
    8. repec:spr:elmark:v:27:y:2017:i:2:d:10.1007_s12525-017-0247-4 is not listed on IDEAS
    9. repec:spr:scient:v:112:y:2017:i:2:d:10.1007_s11192-017-2399-6 is not listed on IDEAS
    10. Simone Righi & Károly Takács, 2017. "The miracle of peer review and development in science: an agent-based model," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 587-607, October.


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:respol:v:42:y:2013:i:1:p:287-294. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Dana Niculescu). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.