IDEAS home Printed from https://ideas.repec.org/p/hhs/kthind/2014_010.html
   My bibliography  Save this paper

Defining the Role of Cognitive Distance in the Peer Review Process: Explorative Study of a Grant Scheme in Infection Biology

Author

Listed:
  • Wang, Qi

    (Department of Industrial Economics and Management, Royal Institute of Technology, Stockholm)

  • Sandström, Ulf

    (Department of Industrial Economics and Management, Royal Institute of Technology, Stockholm)

Abstract

The aim of this paper is twofold: (1) to provide a methodology that can measure cognitive distance between researchers and (2) to explore the role of cognitive distance on the results of peer review processes. Citing references and the content of articles are used to represent their respective scientific knowledge bases. Based on the two different approaches—Author-Bibliographic Coupling analysis and Author-Topic analysis—we apply the methodology on a recent competition for grants from the Swedish Strategic Foundation. Results indicate that cognitive distances between applicants and reviewers might influence peer review results, but that the impact is to some extent at the unexpected end. The main contribution of this paper is the elaboration on the relevance of the concept of cognitive distance to the issue of research evaluation in general, and especially in relation to peer review as a model used in grant decisions.

Suggested Citation

  • Wang, Qi & Sandström, Ulf, 2014. "Defining the Role of Cognitive Distance in the Peer Review Process: Explorative Study of a Grant Scheme in Infection Biology," INDEK Working Paper Series 2014/10, Royal Institute of Technology, Department of Industrial Economics and Management.
    • Handle: RePEc:hhs:kthind:2014_010
    as

    Download full text from publisher

    File URL: https://static.sys.kth.se/itm/wp/indek/indekwp10.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Per O. Seglen, 1994. "Causal relationship between article citedness and journal impact," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 45(1), pages 1-11, January.
    2. Viner, Neil & Powell, Philip & Green, Rod, 2004. "Institutionalized biases in the award of research grants: a preliminary analysis revisiting the principle of accumulative advantage," Research Policy, Elsevier, vol. 33(3), pages 443-454, April.
    3. M. M. Kessler, 1963. "Bibliographic coupling between scientific papers," American Documentation, Wiley Blackwell, vol. 14(1), pages 10-25, January.
    4. Christine Wennerås & Agnes Wold, 1997. "Nepotism and sexism in peer-review," Nature, Nature, vol. 387(6631), pages 341-343, May.
    5. Braun, Dietmar, 1998. "The role of funding agencies in the cognitive development of science," Research Policy, Elsevier, vol. 27(8), pages 807-821, December.
    6. Nooteboom, Bart & Van Haverbeke, Wim & Duysters, Geert & Gilsing, Victor & van den Oord, Ad, 2007. "Optimal cognitive distance and absorptive capacity," Research Policy, Elsevier, vol. 36(7), pages 1016-1034, September.
    7. Ma, Ruimin, 2012. "Author bibliographic coupling analysis: A test based on a Chinese academic database," Journal of Informetrics, Elsevier, vol. 6(4), pages 532-542.
    8. Kevin W. Boyack & Richard Klavans & Katy Börner, 2005. "Mapping the backbone of science," Scientometrics, Springer;Akadémiai Kiadó, vol. 64(3), pages 351-374, August.
    9. H. P. F. Peters & R. R. Braam & A. F. J. van Raan, 1995. "Cognitive resemblance and citation relations in chemical engineering publications," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 46(1), pages 9-21, January.
    10. Cassidy R. Sugimoto & Blaise Cronin, 2013. "Citation gamesmanship: testing for evidence of ego bias in peer review," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(3), pages 851-862, June.
    11. Uwe Cantner & Andreas Meder & Anne ter Wal, 2008. "Innovator networks and regional knowledge base," Jena Economics Research Papers 2008-042, Friedrich-Schiller-University Jena.
    12. Per O. Seglen, 1992. "The skewness of science," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 43(9), pages 628-638, October.
    13. Ahlgren, Per & Colliander, Cristian, 2009. "Document–document similarity approaches and science mapping: Experimental comparison of five approaches," Journal of Informetrics, Elsevier, vol. 3(1), pages 49-63.
    14. Liv Langfeldt, 2006. "The policy challenges of peer review: managing bias, conflict of interests and interdisciplinary assessments," Research Evaluation, Oxford University Press, vol. 15(1), pages 31-41, April.
    15. Bart Nooteboom, 2000. "Learning by Interaction: Absorptive Capacity, Cognitive Distance and Governance," Journal of Management & Governance, Springer;Accademia Italiana di Economia Aziendale (AIDEA), vol. 4(1), pages 69-92, March.
    16. Carole J. Lee & Cassidy R. Sugimoto & Guo Zhang & Blaise Cronin, 2013. "Bias in peer review," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(1), pages 2-17, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Albert Banal-Estañol & Qianshuo Liu & Inés Macho-Stadler & David Pérez-Castrillo, 2021. "Similar-to-me Effects in the Grant Application Process: Applicants, Panelists, and the Likelihood of Obtaining Funds," Working Papers 1289, Barcelona School of Economics.
    2. Banal-Estañol, Albert & Macho-Stadler, Inés & Pérez-Castrillo, David, 2019. "Evaluation in research funding agencies: Are structurally diverse teams biased against?," Research Policy, Elsevier, vol. 48(7), pages 1823-1840.
    3. Kok, Holmer & Faems, Dries & de Faria, Pedro, 2022. "Pork Barrel or Barrel of Gold? Examining the performance implications of earmarking in public R&D grants," Research Policy, Elsevier, vol. 51(7).
    4. Lettice, Fiona & Smart, Palie & Baruch, Yehuda & Johnson, Mark, 2012. "Navigating the impact-innovation double hurdle: The case of a climate change research fund," Research Policy, Elsevier, vol. 41(6), pages 1048-1057.
    5. Andrea Bonaccorsi & Nicola Melluso & Francesco Alessandro Massucci, 2022. "Exploring the antecedents of interdisciplinarity at the European Research Council: a topic modeling approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 6961-6991, December.
    6. Gaëlle Vallée-Tourangeau & Ana Wheelock & Tushna Vandrevala & Priscilla Harries, 2022. "Peer reviewers’ dilemmas: a qualitative exploration of decisional conflict in the evaluation of grant applications in the medical humanities and social sciences," Palgrave Communications, Palgrave Macmillan, vol. 9(1), pages 1-11, December.
    7. Ali Gazni & Fereshteh Didegah, 2016. "The relationship between authors’ bibliographic coupling and citation exchange: analyzing disciplinary differences," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(2), pages 609-626, May.
    8. Jing Zhang & Xiaomin Liu & Lili Wu, 2016. "The study of subject-classification based on journal coupling and expert subject-classification system," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(3), pages 1149-1170, June.
    9. Kevin J. Boudreau & Eva C. Guinan & Karim R. Lakhani & Christoph Riedl, 2016. "Looking Across and Looking Beyond the Knowledge Frontier: Intellectual Distance, Novelty, and Resource Allocation in Science," Management Science, INFORMS, vol. 62(10), pages 2765-2783, October.
    10. A. I. M. Jakaria Rahman & Raf Guns & Loet Leydesdorff & Tim C. E. Engels, 2016. "Measuring the match between evaluators and evaluees: cognitive distances between panel members and research groups at the journal level," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1639-1663, December.
    11. Zhang, Yi & Shang, Lining & Huang, Lu & Porter, Alan L. & Zhang, Guangquan & Lu, Jie & Zhu, Donghua, 2016. "A hybrid similarity measure method for patent portfolio analysis," Journal of Informetrics, Elsevier, vol. 10(4), pages 1108-1130.
    12. Takahiro Kawamura & Katsutaro Watanabe & Naoya Matsumoto & Shusaku Egami & Mari Jibu, 2018. "Funding map using paragraph embedding based on semantic diversity," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(2), pages 941-958, August.
    13. Kaihuang Zhang & Qinglan Qian & Yijing Zhao, 2020. "Evolution of Guangzhou Biomedical Industry Innovation Network Structure and Its Proximity Mechanism," Sustainability, MDPI, vol. 12(6), pages 1-20, March.
    14. Joelle Forest, 2009. "The Creative Rationality As A Key Driver For Enhancing Innovation Capability," Post-Print halshs-00620865, HAL.
    15. van Eck, N.J.P. & Waltman, L., 2009. "How to Normalize Co-Occurrence Data? An Analysis of Some Well-Known Similarity Measures," ERIM Report Series Research in Management ERS-2009-001-LIS, Erasmus Research Institute of Management (ERIM), ERIM is the joint research institute of the Rotterdam School of Management, Erasmus University and the Erasmus School of Economics (ESE) at Erasmus University Rotterdam.
    16. Yu-Wei Chang & Mu-Hsuan Huang & Chiao-Wen Lin, 2015. "Evolution of research subjects in library and information science based on keyword, bibliographical coupling, and co-citation analyses," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 2071-2087, December.
    17. Yang, Siluo & Han, Ruizhen & Wolfram, Dietmar & Zhao, Yuehua, 2016. "Visualizing the intellectual structure of information science (2006–2015): Introducing author keyword coupling analysis," Journal of Informetrics, Elsevier, vol. 10(1), pages 132-150.
    18. Rodríguez Sánchez, Isabel & Makkonen, Teemu & Williams, Allan M., 2019. "Peer review assessment of originality in tourism journals: critical perspective of key gatekeepers," Annals of Tourism Research, Elsevier, vol. 77(C), pages 1-11.
    19. Mariia Shkolnykova & Muhamed Kudic, 2022. "Who benefits from SMEs’ radical innovations?—empirical evidence from German biotechnology," Small Business Economics, Springer, vol. 58(2), pages 1157-1185, February.
    20. Ying Huang & Wolfgang Glänzel & Lin Zhang, 2021. "Tracing the development of mapping knowledge domains," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 6201-6224, July.

    More about this item

    Keywords

    peer review; cognitive distance; cognitive bias;
    All these keywords.

    JEL classification:

    • I20 - Health, Education, and Welfare - - Education - - - General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hhs:kthind:2014_010. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Vardan Hovsepyan (email available below). General contact details of provider: https://edirc.repec.org/data/iikthse.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.