IDEAS home Printed from https://ideas.repec.org/p/bge/wpaper/890.html
   My bibliography  Save this paper

Evaluation in Research Funding Agencies: Are Structurally Diverse Teams Biased Against?

Author

Listed:
  • Albert Banal-Estañol
  • Inés Macho-Stadler
  • David Pérez-Castrillo

Abstract

We analyze whether funding bodies are biased against diverse teams, which have often been linked to the production of transformative research. We develop a general framework that compares the drivers of success in the ex-ante grant decision process to the drivers of success in ex-post performance. We use our framework to systematically analyze the decisions of one of the major public funding organizations for scientific research worldwide, the UK's Engineering and Physical Sciences Research Council (EPSRC). We find that structurally diverse teams are not only penalized but are also biased against. Indeed, although teams that exhibit greater diversity in knowledge and skills, education, and/or scientific ability, are significantly less likely to obtain funding, they are generally more likely to be successful. Our mediating e¤ects show that the evidence of a bias against diversity is weaker for teams led by prestigious researchers.

Suggested Citation

  • Albert Banal-Estañol & Inés Macho-Stadler & David Pérez-Castrillo, 2016. "Evaluation in Research Funding Agencies: Are Structurally Diverse Teams Biased Against?," Working Papers 890, Barcelona School of Economics.
  • Handle: RePEc:bge:wpaper:890
    as

    Download full text from publisher

    File URL: https://www.barcelonagse.eu/sites/default/files/working_paper_pdfs/890_0.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Benjamin F. Jones, 2009. "The Burden of Knowledge and the "Death of the Renaissance Man": Is Innovation Getting Harder?," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(1), pages 283-317.
    2. Terttu Luukkonen, 2012. "Conservatism and risk-taking in peer review: Emerging ERC practices," Research Evaluation, Oxford University Press, vol. 21(1), pages 48-60, February.
    3. Paula Stephan, 2014. "The Endless Frontier: Reaping What Bush Sowed?," NBER Chapters, in: The Changing Frontier: Rethinking Science and Innovation Policy, pages 321-366, National Bureau of Economic Research, Inc.
    4. Wuyts, Stefan & Colombo, Massimo G. & Dutta, Shantanu & Nooteboom, Bart, 2005. "Empirical tests of optimal cognitive distance," Journal of Economic Behavior & Organization, Elsevier, vol. 58(2), pages 277-302, October.
    5. Grimpe, Christoph, 2012. "Extramural research grants and scientists’ funding strategies: Beggars cannot be choosers?," Research Policy, Elsevier, vol. 41(8), pages 1448-1460.
    6. Viner, Neil & Powell, Philip & Green, Rod, 2004. "Institutionalized biases in the award of research grants: a preliminary analysis revisiting the principle of accumulative advantage," Research Policy, Elsevier, vol. 33(3), pages 443-454, April.
    7. Barton H. Hamilton & Jack A. Nickerson & Hideo Owan, 2003. "Team Incentives and Worker Heterogeneity: An Empirical Analysis of the Impact of Teams on Productivity and Participation," Journal of Political Economy, University of Chicago Press, vol. 111(3), pages 465-497, June.
    8. James G. March, 1991. "Exploration and Exploitation in Organizational Learning," Organization Science, INFORMS, vol. 2(1), pages 71-87, February.
    9. Pierre Azoulay & Joshua S. Graff Zivin & Gustavo Manso, 2011. "Incentives and creativity: evidence from the academic life sciences," RAND Journal of Economics, RAND Corporation, vol. 42(3), pages 527-554, September.
    10. Nightingale, Paul, 1998. "A cognitive model of innovation," Research Policy, Elsevier, vol. 27(7), pages 689-709, November.
    11. Karen A. Bantel & Susan E. Jackson, 1989. "Top management and innovations in banking: Does the composition of the top team make a difference?," Strategic Management Journal, Wiley Blackwell, vol. 10(S1), pages 107-124, June.
    12. Christine Wennerås & Agnes Wold, 1997. "Nepotism and sexism in peer-review," Nature, Nature, vol. 387(6631), pages 341-343, May.
    13. Jian Wang & Bart Thijs & Wolfgang Glänzel, 2015. "Interdisciplinarity and Impact: Distinct Effects of Variety, Balance, and Disparity," PLOS ONE, Public Library of Science, vol. 10(5), pages 1-18, May.
    14. Van de Ven, Wynand P. M. M. & Van Praag, Bernard M. S., 1981. "The demand for deductibles in private health insurance : A probit model with sample selection," Journal of Econometrics, Elsevier, vol. 17(2), pages 229-252, November.
    15. Banal-Estañol, Albert & Jofre-Bonet, Mireia & Lawson, Cornelia, 2015. "The double-edged sword of industry collaboration: Evidence from engineering academics in the UK," Research Policy, Elsevier, vol. 44(6), pages 1160-1175.
    16. Mushin Lee & Kiyong Om & Joon Koh, 2000. "The Bias of Sighted Reviewers in Research Proposal Evaluation: A Comparative Analysis of Blind and Open Review in Korea," Scientometrics, Springer;Akadémiai Kiadó, vol. 48(1), pages 99-116, June.
    17. Jonathon N. Cummings, 2004. "Work Groups, Structural Diversity, and Knowledge Sharing in a Global Organization," Management Science, INFORMS, vol. 50(3), pages 352-364, March.
    18. Jasjit Singh & Ajay Agrawal, 2011. "Recruiting for Ideas: How Firms Exploit the Prior Inventions of New Hires," Management Science, INFORMS, vol. 57(1), pages 129-150, January.
    19. Kristina Dahlin & L. Weingart & P. Hinds, 2005. "Team diversity and information use," Post-Print hal-00480406, HAL.
    20. Defazio, Daniela & Lockett, Andy & Wright, Mike, 2009. "Funding incentives, collaborative dynamics and scientific productivity: Evidence from the EU framework program," Research Policy, Elsevier, vol. 38(2), pages 293-305, March.
    21. John-Paul Ferguson & Gianluca Carnabuci, 2017. "Risky Recombinations: Institutional Gatekeeping in the Innovation Process," Organization Science, INFORMS, vol. 28(1), pages 133-151, February.
    22. Sah, Raaj Kumar & Stiglitz, Joseph E, 1986. "The Architecture of Economic Systems: Hierarchies and Polyarchies," American Economic Review, American Economic Association, vol. 76(4), pages 716-727, September.
    23. Philippe Aghion & Mathias Dewatripont & Caroline Hoxby & Andreu Mas-Colell & André Sapir, 2010. "The governance and performance of universities: evidence from Europe and the US [Distance to frontier, selection, and economic growth]," Economic Policy, CEPR, CESifo, Sciences Po;CES;MSH, vol. 25(61), pages 7-59.
    24. Verhoeven, Dennis & Bakker, Jurriën & Veugelers, Reinhilde, 2016. "Measuring technological novelty with patent-based indicators," Research Policy, Elsevier, vol. 45(3), pages 707-723.
    25. Alan L. Porter & Alex S. Cohen & J. David Roessner & Marty Perreault, 2007. "Measuring researcher interdisciplinarity," Scientometrics, Springer;Akadémiai Kiadó, vol. 72(1), pages 117-147, July.
    26. Kevin J. Boudreau & Eva C. Guinan & Karim R. Lakhani & Christoph Riedl, 2016. "Looking Across and Looking Beyond the Knowledge Frontier: Intellectual Distance, Novelty, and Resource Allocation in Science," Management Science, INFORMS, vol. 62(10), pages 2765-2783, October.
    27. Alfredo Yegros-Yegros & Ismael Rafols & Pablo D’Este, 2015. "Does Interdisciplinary Research Lead to Higher Citation Impact? The Different Effect of Proximal and Distal Interdisciplinarity," PLOS ONE, Public Library of Science, vol. 10(8), pages 1-21, August.
    28. Atul Nerkar, 2003. "Old Is Gold? The Value of Temporal Exploration in the Creation of New Knowledge," Management Science, INFORMS, vol. 49(2), pages 211-229, February.
    29. Nelson, Richard R & Winter, Sidney G, 1982. "The Schumpeterian Tradeoff Revisited," American Economic Review, American Economic Association, vol. 72(1), pages 114-132, March.
    30. Albert Banal-Estañol & Mireia Jofre-Bonet & Cornelia Meissner, 2008. "Theimpact of industry collaboration on research: Evidence from engineering academics in the UK," Economics Working Papers 1190, Department of Economics and Business, Universitat Pompeu Fabra, revised Aug 2010.
    31. Robert J. W. Tijssen & Martijn S. Visser & Thed N. van Leeuwen, 2002. "Benchmarking international scientific excellence: Are highly cited research papers an appropriate frame of reference?," Scientometrics, Springer;Akadémiai Kiadó, vol. 54(3), pages 381-397, July.
    32. Loet Leydesdorff & Ismael Rafols, 2009. "A global map of science based on the ISI subject categories," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(2), pages 348-362, February.
    33. Martin L. Weitzman, 1998. "Recombinant Growth," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 113(2), pages 331-360.
    34. Cowan, Robin & Jonard, Nicolas & Zimmermann, J-B, 2004. "Networks as Emergent Structures from Bilateral Collaboration," Research Memorandum 017, Maastricht University, Maastricht Economic Research Institute on Innovation and Technology (MERIT).
    35. Raffaele Conti & Alfonso Gambardella & Myriam Mariani, 2014. "Learning to Be Edison: Inventors, Organizations, and Breakthrough Inventions," Organization Science, INFORMS, vol. 25(3), pages 833-849, June.
    36. Jacob, Brian A. & Lefgren, Lars, 2011. "The impact of research grant funding on scientific productivity," Journal of Public Economics, Elsevier, vol. 95(9), pages 1168-1177.
    37. Seungwha (Andy) Chung & Harbir Singh & Kyungmook Lee, 2000. "Complementarity, status similarity and social capital as drivers of alliance formation," Strategic Management Journal, Wiley Blackwell, vol. 21(1), pages 1-22, January.
    38. Albert Banal-Estañol & Inés Macho-Stadler & David Pérez-Castrillo, 2013. "Research Output From University–Industry Collaborative Projects," Economic Development Quarterly, , vol. 27(1), pages 71-81, February.
    39. Christian Catalini, 2018. "Microgeography and the Direction of Inventive Activity," Management Science, INFORMS, vol. 64(9), pages 4348-4364, September.
    40. Van Looy, Bart & Callaert, Julie & Debackere, Koenraad, 2006. "Publication and patent behavior of academic researchers: Conflicting, reinforcing or merely co-existing?," Research Policy, Elsevier, vol. 35(4), pages 596-608, May.
    41. Lutz Bornmann & Hans-Dieter Daniel, 2005. "Selection of research fellowship recipients by committee peer review. Reliability, fairness and predictive validity of Board of Trustees' decisions," Scientometrics, Springer;Akadémiai Kiadó, vol. 63(2), pages 297-320, April.
    42. Alan L. Porter & Ismael Rafols, 2009. "Is science becoming more interdisciplinary? Measuring and mapping six research fields over time," Scientometrics, Springer;Akadémiai Kiadó, vol. 81(3), pages 719-745, December.
    43. Liv Langfeldt, 2006. "The policy challenges of peer review: managing bias, conflict of interests and interdisciplinary assessments," Research Evaluation, Oxford University Press, vol. 15(1), pages 31-41, April.
    44. Francis Narin & Gabriel Pinski & Helen Hofer Gee, 1976. "Structure of the Biomedical Literature," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 27(1), pages 25-45, January.
    45. Lee, You-Na & Walsh, John P. & Wang, Jian, 2015. "Creativity in scientific teams: Unpacking novelty and impact," Research Policy, Elsevier, vol. 44(3), pages 684-697.
    46. Stefano Breschi & Francesco Lissoni & Fabio Montobbio, 2006. "University patenting and scientific productivity. A quantitative study of Italian academic inventors," KITeS Working Papers 189, KITeS, Centre for Knowledge, Internationalization and Technology Studies, Universita' Bocconi, Milano, Italy, revised Nov 2006.
    47. Giuseppe Scellato & Chiara Franzoni & Paula Stephan, 2012. "Mobile Scientists and International Networks," NBER Working Papers 18613, National Bureau of Economic Research, Inc.
    48. Wagner, Caroline S. & Roessner, J. David & Bobb, Kamau & Klein, Julie Thompson & Boyack, Kevin W. & Keyton, Joann & Rafols, Ismael & Börner, Katy, 2011. "Approaches to understanding and measuring interdisciplinary scientific research (IDR): A review of the literature," Journal of Informetrics, Elsevier, vol. 5(1), pages 14-26.
    49. Danielle Li, 2017. "Expertise versus Bias in Evaluation: Evidence from the NIH," American Economic Journal: Applied Economics, American Economic Association, vol. 9(2), pages 60-92, April.
    50. Albert Banal-Estañol & Inés Macho-Stadler & David Pérez-Castrillo, 2011. "Research Output from University-Industry Collaborative Projects," Working Papers 539, Barcelona Graduate School of Economics.
    51. Jasjit Singh & Lee Fleming, 2010. "Lone Inventors as Sources of Breakthroughs: Myth or Reality?," Management Science, INFORMS, vol. 56(1), pages 41-56, January.
    52. Lee Fleming, 2001. "Recombinant Uncertainty in Technological Search," Management Science, INFORMS, vol. 47(1), pages 117-132, January.
    53. Daniel Ellsberg, 1961. "Risk, Ambiguity, and the Savage Axioms," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 75(4), pages 643-669.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Amara, Nabil & Rhaiem, Mehdi & Halilem, Norrin, 2020. "Assessing the research efficiency of Canadian scholars in the management field: Evidence from the DEA and fsQCA," Journal of Business Research, Elsevier, vol. 115(C), pages 296-306.
    2. Banal-Estañol, Albert & Macho-Stadler, Inés & Nieto-Postigo, Jonás & Pérez-Castrillo, David, 2023. "Early individual stakeholders, first venture capital investment, and exit in the UK startup ecosystem," Journal of Corporate Finance, Elsevier, vol. 80(C).
    3. Lawson, Cornelia & Salter, Ammon, 2023. "Exploring the effect of overlapping institutional applications on panel decision-making," Research Policy, Elsevier, vol. 52(9).
    4. Sila Öcalan-Özel & Patrick Llerena, 2021. "Industry Collaborations of Research Teams: Are They Penalized or Rewarded in the Grant Evaluation Process ?," Post-Print hal-03571919, HAL.
    5. Chiara Franzoni & Paula Stephan & Reinhilde Veugelers, 2022. "Funding Risky Research," Entrepreneurship and Innovation Policy and the Economy, University of Chicago Press, vol. 1(1), pages 103-133.
    6. Albert Banal-Estañol & Inés Macho-Stadler & David Pérez-Castillo, 2019. "Funding academic research: grant application, partnership, award, and output," Economics Working Papers 1658, Department of Economics and Business, Universitat Pompeu Fabra.
    7. Zhang, Chonghui & Jiang, Nanyue & Su, Tiantian & Chen, Ji & Streimikiene, Dalia & Balezentis, Tomas, 2022. "Spreading knowledge and technology: Research efficiency at universities based on the three-stage MCDM-NRSDEA method with bootstrapping," Technology in Society, Elsevier, vol. 68(C).
    8. Nicolas Carayol & Emeric Henry & Marianne Lanoë, 2020. "Stimulating Peer Effects? Evidence from a Research Cluster Policy," Working Papers hal-03874261, HAL.
    9. Mors, Marie Louise & Waguespack, David M., 2021. "Fast success and slow failure: The process speed of dispersed research teams," Research Policy, Elsevier, vol. 50(5).
    10. Abbas Abdul, 2023. "Policy seduction and governance resistance? Examining public funding agencies and academic institutions on decarbonisation research," Science and Public Policy, Oxford University Press, vol. 50(1), pages 87-101.
    11. Ali Sina Önder & Sascha Schweitzer & Hakan Yilmazkuday, 2021. "Field Distance and Quality in Economists’ Collaborations," Working Papers in Economics & Finance 2021-04, University of Portsmouth, Portsmouth Business School, Economics and Finance Subject Group.
    12. Stefano Bianchini & Patrick Llerena & Sıla Öcalan-Özel & Emre Özel, 2022. "Gender diversity of research consortia contributes to funding decisions in a multi-stage grant peer-review process," Palgrave Communications, Palgrave Macmillan, vol. 9(1), pages 1-10, December.
    13. Önder, Ali Sina & Schweitzer, Sascha & Yilmazkuday, Hakan, 2021. "Specialization, field distance, and quality in economists’ collaborations," Journal of Informetrics, Elsevier, vol. 15(4).
    14. Emre Özel, 2024. "What is Gender Bias in Grant Peer review?," Working Papers halshs-03862027, HAL.
    15. Conor O’Kane & Jing A. Zhang & Jarrod Haar & James A. Cunningham, 2023. "How scientists interpret and address funding criteria: value creation and undesirable side effects," Small Business Economics, Springer, vol. 61(2), pages 799-826, August.
    16. Albert Banal-Estañol & Qianshuo Liu & Inés Macho-Stadler & Pérez-Castrillo, 2021. "Similar-to-me effects in the grant application process: Applicants, panelists, and the likelihood of obtaining funds," Economics Working Papers 1801, Department of Economics and Business, Universitat Pompeu Fabra.
    17. Alessandra Allocca, 2023. "“No Man is an Island”: An Empirical Study on Team Formation and Performance," Rationality and Competition Discussion Paper Series 389, CRC TRR 190 Rationality and Competition.
    18. Blandinieres, Florence & Pellens, Maikel, 2021. "Scientist's industry engagement and the research agenda: Evidence from Germany," ZEW Discussion Papers 21-001, ZEW - Leibniz Centre for European Economic Research.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Albert Banal-Estañol & Ines Macho-Stadler & David Pérez-Castrillo, 2016. "Key Success Drivers in Public Research Grants: Funding the Seeds of Radical Innovation in Academia?," CESifo Working Paper Series 5852, CESifo.
    2. Albert Banal-Estañol & Qianshuo Liu & Inés Macho-Stadler & David Pérez-Castrillo, 2021. "Similar-to-me Effects in the Grant Application Process: Applicants, Panelists, and the Likelihood of Obtaining Funds," Working Papers 1289, Barcelona School of Economics.
    3. Albert Banal-Estañol & Inés Macho-Stadler & David Pérez-Castillo, 2019. "Funding academic research: grant application, partnership, award, and output," Economics Working Papers 1658, Department of Economics and Business, Universitat Pompeu Fabra.
    4. Wang, Jian & Veugelers, Reinhilde & Stephan, Paula, 2017. "Bias against novelty in science: A cautionary tale for users of bibliometric indicators," Research Policy, Elsevier, vol. 46(8), pages 1416-1436.
    5. Andrea Bonaccorsi & Nicola Melluso & Francesco Alessandro Massucci, 2022. "Exploring the antecedents of interdisciplinarity at the European Research Council: a topic modeling approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 6961-6991, December.
    6. Fontana, Magda & Iori, Martina & Montobbio, Fabio & Sinatra, Roberta, 2020. "New and atypical combinations: An assessment of novelty and interdisciplinarity," Research Policy, Elsevier, vol. 49(7).
    7. Martin Kalthaus, 2020. "Knowledge recombination along the technology life cycle," Journal of Evolutionary Economics, Springer, vol. 30(3), pages 643-704, July.
    8. Nicolas Carayol, 2016. "The Right Job and the Job Right: Novelty, Impact and Journal Stratification in Science," Post-Print hal-02274661, HAL.
    9. Haeussler, Carolin & Sauermann, Henry, 2020. "Division of labor in collaborative knowledge production: The role of team size and interdisciplinarity," Research Policy, Elsevier, vol. 49(6).
    10. Libo Sheng & Dongqing Lyu & Xuanmin Ruan & Hongquan Shen & Ying Cheng, 2023. "The association between prior knowledge and the disruption of an article," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(8), pages 4731-4751, August.
    11. Orsatti, Gianluca & Quatraro, Francesco & Pezzoni, Michele, 2020. "The antecedents of green technologies: The role of team-level recombinant capabilities," Research Policy, Elsevier, vol. 49(3).
    12. Wang, Jian, 2016. "Knowledge creation in collaboration networks: Effects of tie configuration," Research Policy, Elsevier, vol. 45(1), pages 68-80.
    13. Qing Ke, 2023. "Interdisciplinary research and technological impact: evidence from biomedicine," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(4), pages 2035-2077, April.
    14. Keyvan Vakili & Sarah Kaplan, 2021. "Organizing for innovation: A contingency view on innovative team configuration," Strategic Management Journal, Wiley Blackwell, vol. 42(6), pages 1159-1183, June.
    15. Chen, Shiji & Qiu, Junping & Arsenault, Clément & Larivière, Vincent, 2021. "Exploring the interdisciplinarity patterns of highly cited papers," Journal of Informetrics, Elsevier, vol. 15(1).
    16. Kok, Holmer & Faems, Dries & de Faria, Pedro, 2022. "Pork Barrel or Barrel of Gold? Examining the performance implications of earmarking in public R&D grants," Research Policy, Elsevier, vol. 51(7).
    17. Lawson, Cornelia & Geuna, Aldo & Finardi, Ugo, 2021. "The funding-productivity-gender nexus in science, a multistage analysis," Research Policy, Elsevier, vol. 50(3).
    18. Belkhouja, Mustapha & Fattoum, Senda & Yoon, Hyungseok (David), 2021. "Does greater diversification increase individual productivity? The moderating effect of attention allocation," Research Policy, Elsevier, vol. 50(6).
    19. Kwon, Seokbeom, 2022. "Interdisciplinary knowledge integration as a unique knowledge source for technology development and the role of funding allocation," Technological Forecasting and Social Change, Elsevier, vol. 181(C).
    20. Boeker, Warren & Howard, Michael D. & Basu, Sandip & Sahaym, Arvin, 2021. "Interpersonal relationships, digital technologies, and innovation in entrepreneurial ventures," Journal of Business Research, Elsevier, vol. 125(C), pages 495-507.

    More about this item

    Keywords

    funding organization; diversity; bias; transformative research; research grant;
    All these keywords.

    JEL classification:

    • O32 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Management of Technological Innovation and R&D
    • I23 - Health, Education, and Welfare - - Education - - - Higher Education; Research Institutions

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bge:wpaper:890. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Bruno Guallar (email available below). General contact details of provider: https://edirc.repec.org/data/bargses.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.