IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v14y2020i4s1751157720301188.html
   My bibliography  Save this article

Interpreting CNCIs on a country-scale: The effect of domestic and international collaboration type

Author

Listed:
  • Potter, Ross W.K.
  • Szomszor, Martin
  • Adams, Jonathan

Abstract

Greater collaboration generally produces higher category normalised citation impact (CNCI) and more influential science. Citation differences between domestic and international collaborative articles are known, but obscured in analyses of countries’ CNCIs, compromising evaluation insights. Here, we address this problem by deconstructing and distinguishing domestic and international collaboration types to explore differences in article citation rates between collaboration type and countries. Using Web of Science article data covering 2009–2018, we find that individual country citation and CNCI profiles vary significantly between collaboration types (e.g., domestic single institution and international bilateral) and credit counting methods (full and fractional). The ‘boosting’ effect of international collaboration is greatest where total research capacity is smallest, which could mislead interpretation of performance for policy and management purposes. By incorporating collaboration type into the CNCI calculation, we define a new metric labelled Collab-CNCI. This can account for collaboration effects without presuming credit (as fractional counting does). We recommend that analysts should: (1) partition all article datasets so that citation counts can be normalised by collaboration type (Collab-CNCI) to enable improved interpretation for research policy and management; and (2) consider filtering out smaller entities from multinational and multi-institutional analyses where their inclusion is likely to obscure interpretation.

Suggested Citation

  • Potter, Ross W.K. & Szomszor, Martin & Adams, Jonathan, 2020. "Interpreting CNCIs on a country-scale: The effect of domestic and international collaboration type," Journal of Informetrics, Elsevier, vol. 14(4).
  • Handle: RePEc:eee:infome:v:14:y:2020:i:4:s1751157720301188
    DOI: 10.1016/j.joi.2020.101075
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157720301188
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.joi.2020.101075?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Loet Leydesdorff & Lutz Bornmann & Rüdiger Mutz & Tobias Opthof, 2011. "Turning the tables on citation analysis one more time: Principles for comparing sets of documents," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(7), pages 1370-1381, July.
    2. Adams, Jonathan, 2018. "Information and misinformation in bibliometric time-trend analysis," Journal of Informetrics, Elsevier, vol. 12(4), pages 1063-1071.
    3. Pablo D. Batista & Mônica G. Campiteli & Osame Kinouchi, 2006. "Is it possible to compare researchers with different scientific interests?," Scientometrics, Springer;Akadémiai Kiadó, vol. 68(1), pages 179-189, July.
    4. Leonardo Costa Ribeiro & Márcia Siqueira Rapini & Leandro Alves Silva & Eduardo Motta Albuquerque, 2018. "Growth patterns of the network of international collaboration in science," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(1), pages 159-179, January.
    5. Waltman, Ludo & van Eck, Nees Jan, 2013. "A systematic empirical comparison of different approaches for normalizing citation impact indicators," Journal of Informetrics, Elsevier, vol. 7(4), pages 833-849.
    6. Kaur, Jasleen & Radicchi, Filippo & Menczer, Filippo, 2013. "Universality of scholarly impact metrics," Journal of Informetrics, Elsevier, vol. 7(4), pages 924-932.
    7. Jonathan Adams, 2013. "The fourth age of research," Nature, Nature, vol. 497(7451), pages 557-560, May.
    8. Leo Egghe & Ronald Rousseau & Guido Van Hooydonk, 2000. "Methods for accrediting publications to authors or countries: Consequences for evaluation studies," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 51(2), pages 145-157.
    9. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    10. Jonathan Adams & Karen Gurney & Stuart Marshall, 2007. "Profiling citation impact: A new methodology," Scientometrics, Springer;Akadémiai Kiadó, vol. 72(2), pages 325-344, August.
    11. Mu-Hsuan Huang & Chi-Shiou Lin & Dar-Zen Chen, 2011. "Counting methods, country rank changes, and counting inflation in the assessment of national research productivity and impact," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(12), pages 2427-2436, December.
    12. Miranda, Ruben & Garcia-Carpintero, Esther, 2018. "Overcitation and overrepresentation of review papers in the most cited papers," Journal of Informetrics, Elsevier, vol. 12(4), pages 1015-1030.
    13. Mu‐Hsuan Huang & Chi‐Shiou Lin & Dar‐Zen Chen, 2011. "Counting methods, country rank changes, and counting inflation in the assessment of national research productivity and impact," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(12), pages 2427-2436, December.
    14. Frandsen, Tove Faber & Nicolaisen, Jeppe, 2010. "What is in a name? Credit assignment practices in different disciplines," Journal of Informetrics, Elsevier, vol. 4(4), pages 608-617.
    15. Vincent Larivière & Yves Gingras & Cassidy R. Sugimoto & Andrew Tsou, 2015. "Team size matters: Collaboration and scientific impact since 1900," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(7), pages 1323-1332, July.
    16. Marianne Gauffriau & Peder Olesen Larsen, 2005. "Counting methods are decisive for rankings based on publication and citation studies," Scientometrics, Springer;Akadémiai Kiadó, vol. 64(1), pages 85-93, July.
    17. Waltman, Ludo, 2012. "An empirical analysis of the use of alphabetical authorship in scientific publishing," Journal of Informetrics, Elsevier, vol. 6(4), pages 700-711.
    18. Waltman, Ludo & van Eck, Nees Jan, 2015. "Field-normalized citation impact indicators and the choice of an appropriate counting method," Journal of Informetrics, Elsevier, vol. 9(4), pages 872-894.
    19. Aksnes, Dag W. & Schneider, Jesper W. & Gunnarsson, Magnus, 2012. "Ranking national research systems by citation indicators. A comparative analysis using whole and fractionalised counting methods," Journal of Informetrics, Elsevier, vol. 6(1), pages 36-43.
    20. G. Van Hooydonk, 1997. "Fractional counting of multiauthored publications: Consequences for the impact of authors," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 48(10), pages 944-945, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Nataliya Matveeva & Ivan Sterligov & Andrey Lovakov, 2022. "International scientific collaboration of post-Soviet countries: a bibliometric analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(3), pages 1583-1607, March.
    2. Gordon Rogers & Martin Szomszor & Jonathan Adams, 2020. "Sample size in bibliometric analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 777-794, October.
    3. Claudiu Vasile Kifor & Ana Maria Benedek & Ioan Sîrbu & Roxana Florența Săvescu, 2023. "Institutional drivers of research productivity: a canonical multivariate analysis of Romanian public universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(4), pages 2233-2258, April.
    4. Ba, Zhichao & Mao, Jin & Ma, Yaxue & Liang, Zhentao, 2021. "Exploring the effect of city-level collaboration and knowledge networks on innovation: Evidence from energy conservation field," Journal of Informetrics, Elsevier, vol. 15(3).
    5. Jonathan Adams & Jo Johnson & Jonathan Grant, 2022. "The rise of UK–China research collaboration: Trends, opportunities and challenges [The West Should Start Sending Its Scientists to China]," Science and Public Policy, Oxford University Press, vol. 49(1), pages 132-147.
    6. Maia Chankseliani & Andrey Lovakov & Vladimir Pislyakov, 2021. "A big picture: bibliometric study of academic publications from post-Soviet countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(10), pages 8701-8730, October.
    7. Gangan Prathap, 2023. "Letter to the Editor: Comments on the paper of Safón and Docampo: what are you reading? From core journals to trendy journals in the Library and Information Science (LIS) field," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(7), pages 4137-4142, July.
    8. Vicente Safón & Domingo Docampo, 2023. "What are you reading? From core journals to trendy journals in the Library and Information Science (LIS) field," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(5), pages 2777-2801, May.
    9. Jeffrey Demaine, 2022. "Fractionalization of research impact reveals global trends in university collaboration," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2235-2247, May.
    10. Ross W. K. Potter & Martin Szomszor & Jonathan Adams, 2022. "Comparing standard, collaboration and fractional CNCI at the institutional level: Consequences for performance evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 7435-7448, December.
    11. Zhihong Huang & Qianjin Zong & Xuerui Ji, 2022. "The associations between scientific collaborations of LIS research and its policy impact," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(11), pages 6453-6470, November.
    12. Elizabeth S. Vieira, 2023. "The influence of research collaboration on citation impact: the countries in the European Innovation Scoreboard," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(6), pages 3555-3579, June.
    13. Andrey Lovakov & Maia Chankseliani & Anna Panova, 2022. "Universities vs. research institutes? Overcoming the Soviet legacy of higher education and research," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(11), pages 6293-6313, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    2. Rahman, Mohammad Tariqur & Regenstein, Joe Mac & Kassim, Noor Lide Abu & Haque, Nazmul, 2017. "The need to quantify authors’ relative intellectual contributions in a multi-author paper," Journal of Informetrics, Elsevier, vol. 11(1), pages 275-281.
    3. Pär Sundling, 2023. "Author contributions and allocation of authorship credit: testing the validity of different counting methods in the field of chemical biology," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(5), pages 2737-2762, May.
    4. Osório, António (António Miguel), 2019. "The value and credits of n-authors publications," Working Papers 2072/376026, Universitat Rovira i Virgili, Department of Economics.
    5. Waltman, Ludo & van Eck, Nees Jan, 2015. "Field-normalized citation impact indicators and the choice of an appropriate counting method," Journal of Informetrics, Elsevier, vol. 9(4), pages 872-894.
    6. Ross W. K. Potter & Martin Szomszor & Jonathan Adams, 2022. "Comparing standard, collaboration and fractional CNCI at the institutional level: Consequences for performance evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 7435-7448, December.
    7. Jeffrey Demaine, 2022. "Fractionalization of research impact reveals global trends in university collaboration," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2235-2247, May.
    8. Hagen, Nils T., 2014. "Counting and comparing publication output with and without equalizing and inflationary bias," Journal of Informetrics, Elsevier, vol. 8(2), pages 310-317.
    9. Sandro Tarkhan-Mouravi, 2020. "Traditional indicators inflate some countries’ scientific impact over 10 times," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(1), pages 337-356, April.
    10. Korytkowski, Przemyslaw & Kulczycki, Emanuel, 2019. "Publication counting methods for a national research evaluation exercise," Journal of Informetrics, Elsevier, vol. 13(3), pages 804-816.
    11. Liu, Xuan Zhen & Fang, Hui, 2023. "A geometric counting method adaptive to the author number," Journal of Informetrics, Elsevier, vol. 17(2).
    12. Kim, Jinseok & Kim, Jinmo, 2015. "Rethinking the comparison of coauthorship credit allocation schemes," Journal of Informetrics, Elsevier, vol. 9(3), pages 667-673.
    13. Fairclough, Ruth & Thelwall, Mike, 2015. "National research impact indicators from Mendeley readers," Journal of Informetrics, Elsevier, vol. 9(4), pages 845-859.
    14. Tanel Hirv, 2022. "The interplay of the size of the research system, ways of collaboration, level, and method of funding in determining bibliometric outputs," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(3), pages 1295-1316, March.
    15. Mingers, John & Leydesdorff, Loet, 2015. "A review of theory and practice in scientometrics," European Journal of Operational Research, Elsevier, vol. 246(1), pages 1-19.
    16. Denis Kosyakov & Andrey Guskov, 2022. "Reasons and consequences of changes in Russian research assessment policies," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(8), pages 4609-4630, August.
    17. Siying Li & Huawei Shen & Peng Bao & Xueqi Cheng, 2021. "$$h_u$$ h u -index: a unified index to quantify individuals across disciplines," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(4), pages 3209-3226, April.
    18. António Osório, 2018. "On the impossibility of a perfect counting method to allocate the credits of multi-authored publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 2161-2173, September.
    19. Nykl, Michal & Campr, Michal & Ježek, Karel, 2015. "Author ranking based on personalized PageRank," Journal of Informetrics, Elsevier, vol. 9(4), pages 777-799.
    20. Zaida Chinchilla-Rodríguez & Yi Bu & Nicolás Robinson-García & Cassidy R. Sugimoto, 2021. "An empirical review of the different variants of the probabilistic affinity index as applied to scientific collaboration," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1775-1795, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:14:y:2020:i:4:s1751157720301188. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.