IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0035247.html
   My bibliography  Save this article

Non-Financial Conflicts of Interest in Academic Grant Evaluation: A Qualitative Study of Multiple Stakeholders in France

Author

Listed:
  • Hendy Abdoul
  • Christophe Perrey
  • Florence Tubach
  • Philippe Amiel
  • Isabelle Durand-Zaleski
  • Corinne Alberti

Abstract

Background: Peer review is the most widely used method for evaluating grant applications in clinical research. Criticisms of peer review include lack of equity, suspicion of biases, and conflicts of interest (CoI). CoIs raise questions of fairness, transparency, and trust in grant allocation. Few observational studies have assessed these issues. We report the results of a qualitative study on reviewers’ and applicants’ perceptions and experiences of CoIs in reviews of French academic grant applications. Methodology and Principal Findings: We designed a qualitative study using semi-structured interviews and direct observation. We asked members of assessment panels, external reviewers, and applicants to participate in semi-structured interviews. Two independent researchers conducted in-depth reviews and line-by-line coding of all transcribed interviews, which were also subjected to Tropes® software text analysis, to detect and qualify themes associated with CoIs. Most participants (73/98) spontaneously reported that non-financial CoIs predominated over financial CoIs. Non-financial CoIs mainly involved rivalry among disciplines, cronyism, and geographic and academic biases. However, none of the participants challenged the validity of peer review. Reviewers who felt they might be affected by CoIs said they reacted in a variety of ways: routine refusal to review, routine attempt to conduct an impartial review, or decision on a case-by-case basis. Multiple means of managing non-financial CoIs were suggested, including increased transparency throughout the review process, with public disclosure of non-financial CoIs, and careful selection of independent reviewers, including foreign experts and methodologists. Conclusions: Our study underscores the importance of considering non-financial CoIs when reviewing research grant applications, in addition to financial CoIs. Specific measures are needed to prevent a negative impact of non-financial CoIs on the fairness of resource allocation. Whether and how public disclosure of non-financial CoIs should be accomplished remains debatable.

Suggested Citation

  • Hendy Abdoul & Christophe Perrey & Florence Tubach & Philippe Amiel & Isabelle Durand-Zaleski & Corinne Alberti, 2012. "Non-Financial Conflicts of Interest in Academic Grant Evaluation: A Qualitative Study of Multiple Stakeholders in France," PLOS ONE, Public Library of Science, vol. 7(4), pages 1-10, April.
    • Handle: RePEc:plo:pone00:0035247
    DOI: 10.1371/journal.pone.0035247
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0035247
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0035247&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0035247?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Lutz Bornmann & Gerlind Wallon & Anna Ledin, 2008. "Does the Committee Peer Review Select the Best Applicants for Funding? An Investigation of the Selection Process for Two European Molecular Biology Organization Programmes," PLOS ONE, Public Library of Science, vol. 3(10), pages 1-11, October.
    2. Mushin Lee & Kiyong Om & Joon Koh, 2000. "The Bias of Sighted Reviewers in Research Proposal Evaluation: A Comparative Analysis of Blind and Open Review in Korea," Scientometrics, Springer;Akadémiai Kiadó, vol. 48(1), pages 99-116, June.
    3. The PLoS Medicine Editors, 2008. "Making Sense of Non-Financial Competing Interests," PLOS Medicine, Public Library of Science, vol. 5(9), pages 1-3, September.
    4. Michael L Callaham & John Tercier, 2007. "The Relationship of Previous Training and Experience of Journal Peer Reviewers to Subsequent Review Quality," PLOS Medicine, Public Library of Science, vol. 4(1), pages 1-9, January.
    5. Klaus Jaffe, 2008. "Grants awarded on the basis of political criteria," Nature, Nature, vol. 451(7177), pages 395-395, January.
    6. Bero, Lisa A. Ph.D. & Glantz, Stanton & Hong, Mi-Kyung, 2005. "The Limits of Competing Interest Disclosures," University of California at San Francisco, Center for Tobacco Control Research and Education qt9170d63n, Center for Tobacco Control Research and Education, UC San Francisco.
    7. The PLoS Medicine Editors, 2007. "Peer Review in PLoS Medicine," PLOS Medicine, Public Library of Science, vol. 4(1), pages 1-2, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gaëlle Vallée-Tourangeau & Ana Wheelock & Tushna Vandrevala & Priscilla Harries, 2022. "Peer reviewers’ dilemmas: a qualitative exploration of decisional conflict in the evaluation of grant applications in the medical humanities and social sciences," Palgrave Communications, Palgrave Macmillan, vol. 9(1), pages 1-11, December.
    2. Hendy Abdoul & Christophe Perrey & Philippe Amiel & Florence Tubach & Serge Gottot & Isabelle Durand-Zaleski & Corinne Alberti, 2012. "Peer Review of Grant Applications: Criteria Used and Qualitative Study of Reviewer Practices," PLOS ONE, Public Library of Science, vol. 7(9), pages 1-15, September.
    3. Khaled Shawwa & Romy Kallas & Serge Koujanian & Arnav Agarwal & Ignacio Neumann & Paul Alexander & Kari A O Tikkinen & Gordon Guyatt & Elie A Akl, 2016. "Requirements of Clinical Journals for Authors’ Disclosure of Financial and Non-Financial Conflicts of Interest: A Cross Sectional Study," PLOS ONE, Public Library of Science, vol. 11(3), pages 1-12, March.
    4. Grażyna Wieczorkowska & Katarzyna Kowalczyk, 2021. "Ensuring Sustainable Evaluation: How to Improve Quality of Evaluating Grant Proposals?," Sustainability, MDPI, vol. 13(5), pages 1-11, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hendy Abdoul & Christophe Perrey & Philippe Amiel & Florence Tubach & Serge Gottot & Isabelle Durand-Zaleski & Corinne Alberti, 2012. "Peer Review of Grant Applications: Criteria Used and Qualitative Study of Reviewer Practices," PLOS ONE, Public Library of Science, vol. 7(9), pages 1-15, September.
    2. José Luis Ortega, 2017. "Are peer-review activities related to reviewer bibliometric performance? A scientometric analysis of Publons," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(2), pages 947-962, August.
    3. Jens Jirschitzka & Aileen Oeberst & Richard Göllner & Ulrike Cress, 2017. "Inter-rater reliability and validity of peer reviews in an interdisciplinary field," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 1059-1092, November.
    4. Dennis L Murray & Douglas Morris & Claude Lavoie & Peter R Leavitt & Hugh MacIsaac & Michael E J Masson & Marc-Andre Villard, 2016. "Bias in Research Grant Evaluation Has Dire Consequences for Small Universities," PLOS ONE, Public Library of Science, vol. 11(6), pages 1-19, June.
    5. Effron, Daniel A. & Raj, Medha, 2021. "Disclosing interpersonal conflicts of interest: Revealing whom we like, but not whom we dislike," Organizational Behavior and Human Decision Processes, Elsevier, vol. 164(C), pages 68-85.
    6. Bramoullé, Yann & Orset, Caroline, 2018. "Manufacturing doubt," Journal of Environmental Economics and Management, Elsevier, vol. 90(C), pages 119-133.
    7. Khaled Shawwa & Romy Kallas & Serge Koujanian & Arnav Agarwal & Ignacio Neumann & Paul Alexander & Kari A O Tikkinen & Gordon Guyatt & Elie A Akl, 2016. "Requirements of Clinical Journals for Authors’ Disclosure of Financial and Non-Financial Conflicts of Interest: A Cross Sectional Study," PLOS ONE, Public Library of Science, vol. 11(3), pages 1-12, March.
    8. Feliciani, Thomas & Morreau, Michael & Luo, Junwen & Lucas, Pablo & Shankar, Kalpana, 2022. "Designing grant-review panels for better funding decisions: Lessons from an empirically calibrated simulation model," Research Policy, Elsevier, vol. 51(4).
    9. Bornmann, Lutz & Daniel, Hans-Dieter, 2009. "Extent of type I and type II errors in editorial decisions: A case study on Angewandte Chemie International Edition," Journal of Informetrics, Elsevier, vol. 3(4), pages 348-352.
    10. Eric Libby & Leon Glass, 2010. "The Calculus of Committee Composition," PLOS ONE, Public Library of Science, vol. 5(9), pages 1-8, September.
    11. Kok, Holmer & Faems, Dries & de Faria, Pedro, 2022. "Pork Barrel or Barrel of Gold? Examining the performance implications of earmarking in public R&D grants," Research Policy, Elsevier, vol. 51(7).
    12. Danielle Lee, 2024. "Exploring the determinants of research performance for early-career researchers: a literature review," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(1), pages 181-235, January.
    13. Kevin W. Boyack & Caleb Smith & Richard Klavans, 2018. "Toward predicting research proposal success," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(2), pages 449-461, February.
    14. Bornmann, Lutz & Leydesdorff, Loet & Van den Besselaar, Peter, 2010. "A meta-evaluation of scientific research proposals: Different ways of comparing rejected to awarded applications," Journal of Informetrics, Elsevier, vol. 4(3), pages 211-220.
    15. Gemma Elizabeth Derrick & Alessandra Zimmermann & Helen Greaves & Jonathan Best & Richard Klavans, 2024. "Targeted, actionable and fair: Reviewer reports as feedback and its effect on ECR career choices," Research Evaluation, Oxford University Press, vol. 32(4), pages 648-657.
    16. repec:plo:pone00:0212286 is not listed on IDEAS
    17. Radicchi, Filippo & Castellano, Claudio, 2012. "Testing the fairness of citation indicators for comparison across scientific domains: The case of fractional citation counts," Journal of Informetrics, Elsevier, vol. 6(1), pages 121-130.
    18. T. S. Evans & N. Hopkins & B. S. Kaube, 2012. "Universality of performance indicators based on citation and reference counts," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(2), pages 473-495, November.
    19. Vieira, Elizabeth S. & Cabral, José A.S. & Gomes, José A.N.F., 2014. "How good is a model based on bibliometric indicators in predicting the final decisions made by peers?," Journal of Informetrics, Elsevier, vol. 8(2), pages 390-405.
    20. Dimity Stephen, 2022. "Peer reviewers equally critique theory, method, and writing, with limited effect on the final content of accepted manuscripts," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(6), pages 3413-3435, June.
    21. Irwin Feller, 2013. "Peer review and expert panels as techniques for evaluating the quality of academic research," Chapters, in: Albert N. Link & Nicholas S. Vonortas (ed.), Handbook on the Theory and Practice of Program Evaluation, chapter 5, pages 115-142, Edward Elgar Publishing.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0035247. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.