IDEAS home Printed from https://ideas.repec.org/a/jas/jasssj/2014-74-3.html
   My bibliography  Save this article

Innovation Suppression and Clique Evolution in Peer-Review-Based, Competitive Research Funding Systems: An Agent-Based Model

Author

Listed:
  • Pawel Sobkowicz

Abstract

Peer review is ubiquitous in modern science: from the evaluation of publications to the distribution of funding. While there is a long tradition of, and many arguments for, peer review as a beneficial and necessary component of scientific processes, the exponential growth of the research community, the ‘publish or perish’ pressures and increasing insecurity and competition for research grants have led to an increasing number of voices describing the weaknesses of the system. One of the most frequent accusations against the peer review system is that it inhibits true innovation. The availability of better data mining tools allows interested stakeholders, in principle, to monitor many aspects of the process and to promote a better understanding of the interplay of various factors. ‘In principle’ – because a lot of information is hidden behind the screens of anonymity and confidentiality. Our work presents an attempt at a theoretical understanding of some aspects of the process via an idealized agent-based model, which describes the effects of the peer review done by ‘imperfect’ agents, in particular with respect to promotion of mediocrity and to formation of self-serving cliques. The results of the model suggest that both phenomena can be quite robust and require careful monitoring of the system to combat their negative effects. Some mitigating measures are simulated and discussed.

Suggested Citation

  • Pawel Sobkowicz, 2015. "Innovation Suppression and Clique Evolution in Peer-Review-Based, Competitive Research Funding Systems: An Agent-Based Model," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 18(2), pages 1-13.
  • Handle: RePEc:jas:jasssj:2014-74-3
    as

    Download full text from publisher

    File URL: https://www.jasss.org/18/2/13/13.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Grimpe, Christoph, 2012. "Extramural research grants and scientists’ funding strategies: Beggars cannot be choosers?," Research Policy, Elsevier, vol. 41(8), pages 1448-1460.
    2. Christine Wennerås & Agnes Wold, 1997. "Nepotism and sexism in peer-review," Nature, Nature, vol. 387(6631), pages 341-343, May.
    3. Mario Paolucci & Francisco Grimaldo, 2014. "Mechanism change in a simulation of peer review: from junk support to elitism," Scientometrics, Springer;Akadémiai Kiadó, vol. 99(3), pages 663-688, June.
    4. Thomas Scherngell & Rafael Lata, 2013. "Towards an integrated European Research Area? Findings from Eigenvector spatially filtered spatial interaction models using European Framework Programme data," Papers in Regional Science, Wiley Blackwell, vol. 92(3), pages 555-577, August.
    5. Stephan, Paula E., 2010. "The Economics of Science," Handbook of the Economics of Innovation, in: Bronwyn H. Hall & Nathan Rosenberg (ed.), Handbook of the Economics of Innovation, edition 1, volume 1, chapter 0, pages 217-273, Elsevier.
    6. Squazzoni, Flaminio & Gandelli, Claudio, 2012. "Saint Matthew strikes again: An agent-based model of peer review and the scientific community structure," Journal of Informetrics, Elsevier, vol. 6(2), pages 265-275.
    7. Abrahamson, Eric, 2009. "Necessary conditions for the study of fads and fashions in science," Scandinavian Journal of Management, Elsevier, vol. 25(2), pages 235-239, June.
    8. S. Thurner & R. Hanel, 2011. "Peer-review in a world with rational scientists: Toward selection of the average," The European Physical Journal B: Condensed Matter and Complex Systems, Springer;EDP Sciences, vol. 84(4), pages 707-711, December.
    9. Flaminio Squazzoni & Claudio Gandelli, 2013. "Opening the Black-Box of Peer Review: An Agent-Based Model of Scientist Behaviour," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 16(2), pages 1-3.
    10. Starbuck, William H., 2009. "The constant causes of never-ending faddishness in the behavioral and social sciences," Scandinavian Journal of Management, Elsevier, vol. 25(1), pages 108-116, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Thomas Feliciani & Junwen Luo & Lai Ma & Pablo Lucas & Flaminio Squazzoni & Ana Marušić & Kalpana Shankar, 2019. "A scoping review of simulation models of peer review," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(1), pages 555-594, October.
    2. Simone Righi & Károly Takács, 2017. "The miracle of peer review and development in science: an agent-based model," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 587-607, October.
    3. Simone Righi & Karoly Takacs, 2016. "The Miracle of Peer Review and Development inScience: An Agent-Based Model," Department of Economics 0090, University of Modena and Reggio E., Faculty of Economics "Marco Biagi".
    4. A. I. M. Jakaria Rahman & Raf Guns & Loet Leydesdorff & Tim C. E. Engels, 2016. "Measuring the match between evaluators and evaluees: cognitive distances between panel members and research groups at the journal level," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1639-1663, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Thomas Feliciani & Junwen Luo & Lai Ma & Pablo Lucas & Flaminio Squazzoni & Ana Marušić & Kalpana Shankar, 2019. "A scoping review of simulation models of peer review," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(1), pages 555-594, October.
    2. Federico Bianchi & Francisco Grimaldo & Giangiacomo Bravo & Flaminio Squazzoni, 2018. "The peer review game: an agent-based model of scientists facing resource constraints and institutional pressures," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 1401-1420, September.
    3. Michail Kovanis & Ludovic Trinquart & Philippe Ravaud & Raphaël Porcher, 2017. "Evaluating alternative systems of peer review: a large-scale agent-based modelling approach to scientific publication," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 651-671, October.
    4. Michail Kovanis & Raphaël Porcher & Philippe Ravaud & Ludovic Trinquart, 2016. "Complex systems approach to scientific publication and peer-review system: development of an agent-based model calibrated with empirical journal data," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(2), pages 695-715, February.
    5. Albert Banal-Estañol & Qianshuo Liu & Inés Macho-Stadler & David Pérez-Castrillo, 2021. "Similar-to-me Effects in the Grant Application Process: Applicants, Panelists, and the Likelihood of Obtaining Funds," Working Papers 1289, Barcelona School of Economics.
    6. J. A. Garcia & Rosa Rodriguez-Sánchez & J. Fdez-Valdivia, 2021. "The interplay between the reviewer’s incentives and the journal’s quality standard," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(4), pages 3041-3061, April.
    7. Banal-Estañol, Albert & Macho-Stadler, Inés & Pérez-Castrillo, David, 2019. "Evaluation in research funding agencies: Are structurally diverse teams biased against?," Research Policy, Elsevier, vol. 48(7), pages 1823-1840.
    8. Iris Wanzenböck & Thomas Scherngell & Thomas Brenner, 2014. "Embeddedness of regions in European knowledge networks: a comparative analysis of inter-regional R&D collaborations, co-patents and co-publications," The Annals of Regional Science, Springer;Western Regional Science Association, vol. 53(2), pages 337-368, September.
    9. Feliciani, Thomas & Morreau, Michael & Luo, Junwen & Lucas, Pablo & Shankar, Kalpana, 2022. "Designing grant-review panels for better funding decisions: Lessons from an empirically calibrated simulation model," Research Policy, Elsevier, vol. 51(4).
    10. Damien Besancenot & Radu Vranceanu, 2015. "Fear Of Novelty: A Model Of Scientific Discovery With Strategic Uncertainty," Economic Inquiry, Western Economic Association International, vol. 53(2), pages 1132-1139, April.
    11. Damien Besancenot & Habib Dogguy, 2011. "Paradigm Shift," Working Papers halshs-00590527, HAL.
    12. Francisco Grimaldo & Mario Paolucci & Jordi Sabater-Mir, 2018. "Reputation or peer review? The role of outliers," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 1421-1438, September.
    13. Hottenrott, Hanna & Lawson, Cornelia, 2013. "Fishing for Complementarities: Competitive Research Funding and Research Productivity," Department of Economics and Statistics Cognetti de Martiis LEI & BRICK - Laboratory of Economics of Innovation "Franco Momigliano", Bureau of Research in Innovation, Complexity and Knowledge, Collegio 201318, University of Turin.
    14. Albert Banal-Estañol & Inés Macho-Stadler & David Pérez-Castillo, 2019. "Funding academic research: grant application, partnership, award, and output," Economics Working Papers 1658, Department of Economics and Business, Universitat Pompeu Fabra.
    15. J. A. Garcia & Rosa Rodriguez-Sánchez & J. Fdez-Valdivia, 2020. "The author–reviewer game," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(3), pages 2409-2431, September.
    16. Simone Righi & Károly Takács, 2017. "The miracle of peer review and development in science: an agent-based model," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 587-607, October.
    17. Bravo, Giangiacomo & Farjam, Mike & Grimaldo Moreno, Francisco & Birukou, Aliaksandr & Squazzoni, Flaminio, 2018. "Hidden connections: Network effects on editorial decisions in four computer science journals," Journal of Informetrics, Elsevier, vol. 12(1), pages 101-112.
    18. Lawson, Cornelia & Geuna, Aldo & Finardi, Ugo, 2021. "The funding-productivity-gender nexus in science, a multistage analysis," Research Policy, Elsevier, vol. 50(3).
    19. Materia, V.C. & Pascucci, S. & Kolympiris, C., 2015. "Understanding the selection processes of public research projects in agriculture: The role of scientific merit," Food Policy, Elsevier, vol. 56(C), pages 87-99.
    20. Hottenrott, Hanna & Lawson, Cornelia, 2017. "Fishing for complementarities: Research grants and research productivity," International Journal of Industrial Organization, Elsevier, vol. 51(C), pages 1-38.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:jas:jasssj:2014-74-3. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Francesco Renzini (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.