IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v13y2019i4s1751157719300355.html
   My bibliography  Save this article

A Game Theoretic Approach to Peer Review of Grant Proposals

Author

Listed:
  • Bayindir, Esra Eren
  • Gurdal, Mehmet Yigit
  • Saglam, Ismail

Abstract

This paper studies the grant peer review process employed by the Turkish regional development agencies, which is adapted from a review procedure of the Education, Audiovisual and Culture Executive Agency of the European Union. To model this process, we consider a Bayesian strategic-form game played by three reviewers who observe both a common and a private score signal about an evaluated project and assign their scores to minimize the sum of their disutilities from the false acceptance and false rejection of the project. We numerically compute the Bayesian Nash equilibria of this game and conduct several comparative statics exercises, after calibrating the model parameters accordingly. We also introduce two simpler review processes and compare their performances to that of the calibrated process in terms of outcome statistics, involving pass and fail rates of the evaluated projects, and manipulation statistics, involving the reviewers’ manipulation rate and size of scores.

Suggested Citation

  • Bayindir, Esra Eren & Gurdal, Mehmet Yigit & Saglam, Ismail, 2019. "A Game Theoretic Approach to Peer Review of Grant Proposals," Journal of Informetrics, Elsevier, vol. 13(4).
    • Handle: RePEc:eee:infome:v:13:y:2019:i:4:s1751157719300355
    DOI: 10.1016/j.joi.2019.100981
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157719300355
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.joi.2019.100981?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Gerling, Kerstin & Gruner, Hans Peter & Kiel, Alexandra & Schulte, Elisabeth, 2005. "Information acquisition and decision making in committees: A survey," European Journal of Political Economy, Elsevier, vol. 21(3), pages 563-597, September.
    2. Mohamed N. Jouini & Robert T. Clemen, 1996. "Copula Models for Aggregating Expert Opinions," Operations Research, INFORMS, vol. 44(3), pages 444-457, June.
    3. Upali W. Jayasinghe & Herbert W. Marsh & Nigel Bond, 2003. "A multilevel cross‐classified modelling approach to peer review of grant proposals: the effects of assessor and researcher attributes on assessor ratings," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 166(3), pages 279-300, October.
    4. Alfredo Altuzarra & José María Moreno-Jiménez & Manuel Salvador, 2010. "Consensus Building in AHP-Group Decision Making: A Bayesian Approach," Operations Research, INFORMS, vol. 58(6), pages 1755-1773, December.
    5. Feddersen, Timothy & Pesendorfer, Wolfgang, 1998. "Convicting the Innocent: The Inferiority of Unanimous Jury Verdicts under Strategic Voting," American Political Science Review, Cambridge University Press, vol. 92(1), pages 23-35, March.
    6. repec:dau:papers:123456789/15122 is not listed on IDEAS
    7. Michel Balinski & Rida Laraki, 2014. "Judge: Don't Vote !," Operations Research, INFORMS, vol. 62(3), pages 483-511, June.
    8. Hao Li & Sherwin Rosen & Wing Suen, 2001. "Conflicts and Common Interests in Committees," American Economic Review, American Economic Association, vol. 91(5), pages 1478-1497, December.
    9. P. M. Hartigan, 1985. "Computation of the Dip Statistic to Test for Unimodality," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 34(3), pages 320-325, November.
    10. Nicholas G. Hall & John C. Hershey & Larry G. Kessler & R. Craig Stotts, 1992. "A Model for Making Project Funding Decisions at the National Cancer Institute," Operations Research, INFORMS, vol. 40(6), pages 1040-1052, December.
    11. Austen-Smith, David & Banks, Jeffrey S., 1996. "Information Aggregation, Rationality, and the Condorcet Jury Theorem," American Political Science Review, Cambridge University Press, vol. 90(1), pages 34-45, March.
    12. David G Pina & Darko Hren & Ana Marušić, 2015. "Peer Review Evaluation Process of Marie Curie Actions under EU’s Seventh Framework Programme for Research," PLOS ONE, Public Library of Science, vol. 10(6), pages 1-15, June.
    13. R. Winkler & Javier Muñoz & José Cervera & José Bernardo & Gail Blattenberger & Joseph Kadane & Dennis Lindley & Allan Murphy & Robert Oliver & David Ríos-Insua, 1996. "Scoring rules and the evaluation of probabilities," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 5(1), pages 1-60, June.
    14. Victor Richmond R. Jose & Robert L. Winkler, 2009. "Evaluating Quantile Assessments," Operations Research, INFORMS, vol. 57(5), pages 1287-1297, October.
    15. J. A. García & Rosa Rodriguez-Sánchez & J. Fdez-Valdivia, 2015. "The author–editor game," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(1), pages 361-380, July.
    16. Rüdiger Mutz & Lutz Bornmann & Hans-Dieter Daniel, 2012. "Heterogeneity of Inter-Rater Reliabilities of Grant Peer Reviews and Its Determinants: A General Estimating Equations Approach," PLOS ONE, Public Library of Science, vol. 7(10), pages 1-10, October.
    17. J. A. García & Rosa Rodriguez-Sánchez & J. Fdez-Valdivia, 2016. "Authors and reviewers who suffer from confirmatory bias," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1377-1395, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Hren, Darko & Pina, David G. & Norman, Christopher R. & Marušić, Ana, 2022. "What makes or breaks competitive research proposals? A mixed-methods analysis of research grant evaluation reports," Journal of Informetrics, Elsevier, vol. 16(2).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Stephen Hansen & Michael McMahon, 2016. "First Impressions Matter: Signalling as a Source of Policy Dynamics," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 83(4), pages 1645-1672.
    2. Chan, Jimmy & Gupta, Seher & Li, Fei & Wang, Yun, 2019. "Pivotal persuasion," Journal of Economic Theory, Elsevier, vol. 180(C), pages 178-202.
      • Jimmy Chan & Seher Gupta & Fei Li & Yun Wang, 2018. "Pivotal Persuasion," Working Papers 2018-11-03, Wang Yanan Institute for Studies in Economics (WISE), Xiamen University.
    3. Kim, Jaehoon & Fey, Mark, 2007. "The swing voter's curse with adversarial preferences," Journal of Economic Theory, Elsevier, vol. 135(1), pages 236-252, July.
    4. Hongbin Cai, 2009. "Costly participation and heterogeneous preferences in informational committees," RAND Journal of Economics, RAND Corporation, vol. 40(1), pages 173-189, March.
    5. Li Hao & Wing Suen, 2009. "Viewpoint: Decision-making in committees," Canadian Journal of Economics, Canadian Economics Association, vol. 42(2), pages 359-392, May.
    6. Breitmoser, Yves & Valasek, Justin, 2017. "A rationale for unanimity in committees," Discussion Papers, Research Unit: Economics of Change SP II 2017-308, WZB Berlin Social Science Center.
    7. Volker Hahn, 2012. "On the Optimal Size of Committees of Experts," Working Paper Series of the Department of Economics, University of Konstanz 2012-24, Department of Economics, University of Konstanz.
    8. Elisabeth Schulte, 2010. "Information aggregation and preference heterogeneity in committees," Theory and Decision, Springer, vol. 69(1), pages 97-118, July.
    9. Xie, Yinxi & Xie, Yang, 2017. "Machiavellian experimentation," Journal of Comparative Economics, Elsevier, vol. 45(4), pages 685-711.
    10. Ruth Ben-Yashar & Igal Milchtaich, 2003. "First and Second Best Voting Rules in Committees," Working Papers 2003-08, Bar-Ilan University, Department of Economics.
    11. Iaryczower, Matias & Lewis, Garrett & Shum, Matthew, 2013. "To elect or to appoint? Bias, information, and responsiveness of bureaucrats and politicians," Journal of Public Economics, Elsevier, vol. 97(C), pages 230-244.
    12. Alvaro J. Name-Correa & Huseyin Yildirim, 2018. "A capture theory of committees," Public Choice, Springer, vol. 177(1), pages 135-154, October.
    13. Gerling, Kerstin & Gruner, Hans Peter & Kiel, Alexandra & Schulte, Elisabeth, 2005. "Information acquisition and decision making in committees: A survey," European Journal of Political Economy, Elsevier, vol. 21(3), pages 563-597, September.
    14. Sourav Bhattacharya, 2006. "Preference Monotonicity and Information Aggregation in Elections," Working Paper 325, Department of Economics, University of Pittsburgh, revised Dec 2008.
    15. Gershkov, Alex & Szentes, Balázs, 2009. "Optimal voting schemes with costly information acquisition," Journal of Economic Theory, Elsevier, vol. 144(1), pages 36-68, January.
    16. Hansen, Stephen & McMahon, Michael, 2010. "What Do Outside Experts Bring To A Committee? Evidence From The Bank of England," Economic Research Papers 270783, University of Warwick - Department of Economics.
    17. Rosar, Frank, 2015. "Continuous decisions by a committee: Median versus average mechanisms," Journal of Economic Theory, Elsevier, vol. 159(PA), pages 15-65.
    18. Hyoungsik Noh, 2023. "Conservativeness in jury decision-making," Theory and Decision, Springer, vol. 95(1), pages 151-172, July.
    19. Arnab Bhattacharjee & Sean Holly, 2013. "Understanding Interactions in Social Networks and Committees," Spatial Economic Analysis, Taylor & Francis Journals, vol. 8(1), pages 23-53, March.
    20. Gersbach, Hans & Mamageishvili, Akaki & Tejada, Oriol, 2020. "Appointed Learning for the Common Good: Optimal Committee Size and Efficient Rewards," CEPR Discussion Papers 15311, C.E.P.R. Discussion Papers.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:13:y:2019:i:4:s1751157719300355. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.