IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v15y2021i4s1751157721000729.html
   My bibliography  Save this article

A network embedding-based scholar assessment indicator considering four facets: Research topic, author credit allocation, field-normalized journal impact, and published time

Author

Listed:
  • Xie, Qing
  • Zhang, Xinyuan
  • Song, Min

Abstract

Scholar performance assessment plays an important role in reward evaluation, funding allocation, and promotion and recruitment decisions. However, raw publication counts and raw citation count-based scholar performance assessment indicators, such as H-index or author citations, have shortcomings; for example, they ignore the impact of different citation patterns under different research topics, leading to authorship credit inflation due to full citation allocation to each author in multi-author publications. This study proposes a new scholar performance assessment indicator called the normalized scholar academic productivity (NSAP) indicator, which overcomes the issues posed by raw citation counts and publication counts-related scholar performance indicators by considering diverse aspects of scholar research achievements. The NSAP indicator considers the research topic, author sequence and author role in the author list, field-normalized journal impact when allocating citation counts to scholars, and published time. The research topic is generated by the co-keyword embedding and semantic relatedness of each keyword in order to make NSAP topic-dependent; the author sequence and role affect authorship credit allocation strategy; and field-normalized journal impact was used to assign different weights on raw publication counts and citation counts. Finally, awardees of the Derek de Solla Price Memorial Medal and the Association for Information Science and Technology's awards were used to evaluate the validity of NSAP for calculating scholar performance assessment. Results reveal outstanding topic-related scholar performance assessment properties compared to raw citation count indicators, such as H-index, author citations, and cited-by counts (i.e., total number of citing authors).

Suggested Citation

  • Xie, Qing & Zhang, Xinyuan & Song, Min, 2021. "A network embedding-based scholar assessment indicator considering four facets: Research topic, author credit allocation, field-normalized journal impact, and published time," Journal of Informetrics, Elsevier, vol. 15(4).
  • Handle: RePEc:eee:infome:v:15:y:2021:i:4:s1751157721000729
    DOI: 10.1016/j.joi.2021.101201
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157721000729
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.joi.2021.101201?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Staša Milojević & Cassidy R. Sugimoto & Erjia Yan & Ying Ding, 2011. "The cognitive structure of Library and Information Science: Analysis of article title words," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(10), pages 1933-1953, October.
    2. Robert R. Braam & Henk F. Moed & Anthony F. J. van Raan, 1991. "Mapping of science by combined co‐citation and word analysis. I. Structural aspects," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 42(4), pages 233-251, May.
    3. Rahman, Mohammad Tariqur & Regenstein, Joe Mac & Kassim, Noor Lide Abu & Haque, Nazmul, 2017. "The need to quantify authors’ relative intellectual contributions in a multi-author paper," Journal of Informetrics, Elsevier, vol. 11(1), pages 275-281.
    4. Zhang, Yi & Lu, Jie & Liu, Feng & Liu, Qian & Porter, Alan & Chen, Hongshu & Zhang, Guangquan, 2018. "Does deep learning help topic extraction? A kernel k-means clustering method with word embedding," Journal of Informetrics, Elsevier, vol. 12(4), pages 1099-1117.
    5. Li, Yongli & Wu, Chong & Wang, Xiaoyu & Luo, Peng, 2014. "A network-based and multi-parameter model for finding influential authors," Journal of Informetrics, Elsevier, vol. 8(3), pages 791-799.
    6. Thelwall, Mike & Fairclough, Ruth, 2015. "The influence of time and discipline on the magnitude of correlations between citation counts and quality scores," Journal of Informetrics, Elsevier, vol. 9(3), pages 529-541.
    7. Johan Bollen & Herbert Van de Sompel & Aric Hagberg & Ryan Chute, 2009. "A Principal Component Analysis of 39 Scientific Impact Measures," PLOS ONE, Public Library of Science, vol. 4(6), pages 1-11, June.
    8. Fuli Zhang, 2017. "Evaluating journal impact based on weighted citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 1155-1169, November.
    9. Bornmann, Lutz & Williams, Richard, 2013. "How to calculate the practical significance of citation impact differences? An empirical example from evaluative institutional bibliometrics using adjusted predictions and marginal effects," Journal of Informetrics, Elsevier, vol. 7(2), pages 562-574.
    10. Hagen, Nils T., 2013. "Harmonic coauthor credit: A parsimonious quantification of the byline hierarchy," Journal of Informetrics, Elsevier, vol. 7(4), pages 784-791.
    11. Behrouzi, Saman & Shafaeipour Sarmoor, Zahra & Hajsadeghi, Khosrow & Kavousi, Kaveh, 2020. "Predicting scientific research trends based on link prediction in keyword networks," Journal of Informetrics, Elsevier, vol. 14(4).
    12. Robert R. Braam & Henk F. Moed & Anthony F. J. van Raan, 1991. "Mapping of science by combined co‐citation and word analysis. II: Dynamical aspects," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 42(4), pages 252-266, May.
    13. Lutz Bornmann & Werner Marx & Andreas Barth, 2013. "The Normalization of Citation Counts Based on Classification Systems," Publications, MDPI, vol. 1(2), pages 1-9, August.
    14. Leo Egghe & Ronald Rousseau & Guido Van Hooydonk, 2000. "Methods for accrediting publications to authors or countries: Consequences for evaluation studies," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 51(2), pages 145-157.
    15. Nadia Simoes & Nuno Crespo, 2020. "A flexible approach for measuring author-level publishing performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(1), pages 331-355, January.
    16. Hans-Dieter Daniel, 2019. "Lutz Bornmann: Recipient of the 2019 Derek John de Solla Price Medal," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(3), pages 1235-1238, December.
    17. Leo Egghe, 2006. "Theory and practise of the g-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 69(1), pages 131-152, October.
    18. Andreas Strotmann & Dangzhi Zhao, 2012. "Author name disambiguation: What difference does it make in author-based citation analysis?," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(9), pages 1820-1833, September.
    19. Xuan Zhen Liu & Hui Fang, 2012. "Fairly sharing the credit of multi-authored papers and its application in the modification of h-index and g-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(1), pages 37-49, April.
    20. Jaime A. Teixeira da Silva & Aamir Raoof Memon, 2017. "CiteScore: A cite for sore eyes, or a valuable, transparent metric?," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(1), pages 553-556, April.
    21. Jinseok Kim & Jana Diesner, 2014. "A network-based approach to coauthorship credit allocation," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 587-602, October.
    22. Michel Zitt & Suzy Ramanana-Rahary & Elise Bassecoulard, 2005. "Relativity of citation performance and excellence measures: From cross-field to cross-scale effects of field-normalisation," Scientometrics, Springer;Akadémiai Kiadó, vol. 63(2), pages 373-401, April.
    23. Ronald Rousseau, 2004. "Loet Leydesdorff: Recipient of the 2003 Derek de Solla Price Award," Scientometrics, Springer;Akadémiai Kiadó, vol. 60(3), pages 275-277, August.
    24. Nils T. Hagen, 2010. "Harmonic publication and citation counting: sharing authorship credit equitably – not equally, geometrically or arithmetically," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(3), pages 785-793, September.
    25. Andreas Strotmann & Dangzhi Zhao, 2012. "Author name disambiguation: What difference does it make in author‐based citation analysis?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(9), pages 1820-1833, September.
    26. Staša Milojević & Cassidy R. Sugimoto & Erjia Yan & Ying Ding, 2011. "The cognitive structure of Library and Information Science: Analysis of article title words," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(10), pages 1933-1953, October.
    27. N. Assimakis & M. Adam, 2010. "A new author’s productivity index: p-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(2), pages 415-427, November.
    28. Olle Persson, 2001. "All author citations versus first author citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 50(2), pages 339-344, February.
    29. Frank J. Trueba & Héctor Guerrero, 2004. "A robust formula to credit authors for their publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 60(2), pages 181-204, June.
    30. Moed, Henk F., 2010. "Measuring contextual citation impact of scientific journals," Journal of Informetrics, Elsevier, vol. 4(3), pages 265-277.
    31. Haunschild, Robin & Leydesdorff, Loet & Bornmann, Lutz & Hellsten, Iina & Marx, Werner, 2019. "Does the public discuss other topics on climate change than researchers? A comparison of explorative networks based on author keywords and hashtags," Journal of Informetrics, Elsevier, vol. 13(2), pages 695-707.
    32. António Osório, 2018. "On the impossibility of a perfect counting method to allocate the credits of multi-authored publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 2161-2173, September.
    33. Kim, Jinseok & Kim, Jinmo, 2015. "Rethinking the comparison of coauthorship credit allocation schemes," Journal of Informetrics, Elsevier, vol. 9(3), pages 667-673.
    34. Waltman, Ludo, 2012. "An empirical analysis of the use of alphabetical authorship in scientific publishing," Journal of Informetrics, Elsevier, vol. 6(4), pages 700-711.
    35. Eom, Sean, 2008. "All author cocitation analysis and first author cocitation analysis: A comparative empirical investigation," Journal of Informetrics, Elsevier, vol. 2(1), pages 53-64.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Xie, Qing & Zhang, Xinyuan & Kim, Giyeong & Song, Min, 2022. "Exploring the influence of coauthorship with top scientists on researchers’ affiliation, research topic, productivity, and impact," Journal of Informetrics, Elsevier, vol. 16(3).
    2. Zhai, Li & Yan, Xiangbin, 2022. "A directed collaboration network for exploring the order of scientific collaboration," Journal of Informetrics, Elsevier, vol. 16(4).
    3. Xinyuan Zhang & Qing Xie & Chaemin Song & Min Song, 2022. "Mining the evolutionary process of knowledge through multiple relationships between keywords," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(4), pages 2023-2053, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    2. António Osório, 2018. "On the impossibility of a perfect counting method to allocate the credits of multi-authored publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 2161-2173, September.
    3. Osório, António (António Miguel), 2019. "The value and credits of n-authors publications," Working Papers 2072/376026, Universitat Rovira i Virgili, Department of Economics.
    4. Liu, Xuan Zhen & Fang, Hui, 2023. "A geometric counting method adaptive to the author number," Journal of Informetrics, Elsevier, vol. 17(2).
    5. Pär Sundling, 2023. "Author contributions and allocation of authorship credit: testing the validity of different counting methods in the field of chemical biology," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(5), pages 2737-2762, May.
    6. Kim, Jinseok & Kim, Jinmo, 2015. "Rethinking the comparison of coauthorship credit allocation schemes," Journal of Informetrics, Elsevier, vol. 9(3), pages 667-673.
    7. Zhai, Li & Yan, Xiangbin, 2022. "A directed collaboration network for exploring the order of scientific collaboration," Journal of Informetrics, Elsevier, vol. 16(4).
    8. Susan George & Hiran H. Lathabai & Thara Prabhakaran & Manoj Changat, 2020. "A framework towards bias-free contextual productivity assessment," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(1), pages 127-157, January.
    9. Jinseok Kim & Jana Diesner, 2014. "A network-based approach to coauthorship credit allocation," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 587-602, October.
    10. Xuan Zhen Liu & Hui Fang, 2014. "Scientific group leaders’ authorship preferences: an empirical investigation," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(2), pages 909-925, February.
    11. Zhang, Fang & Wu, Shengli, 2020. "Predicting future influence of papers, researchers, and venues in a dynamic academic network," Journal of Informetrics, Elsevier, vol. 14(2).
    12. Xuan Zhen Liu & Hui Fang, 2014. "The impact of publications from mainland China on the trends in alphabetical authorship," Scientometrics, Springer;Akadémiai Kiadó, vol. 99(3), pages 865-879, June.
    13. Jingda Ding & Chao Liu & Qiao Zheng & Wei Cai, 2021. "A new method of co-author credit allocation based on contributor roles taxonomy: proof of concept and evaluation using papers published in PLOS ONE," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 7561-7581, September.
    14. Hagen, Nils T., 2014. "Reversing the byline hierarchy: The effect of equalizing bias on the accreditation of primary, secondary and senior authors," Journal of Informetrics, Elsevier, vol. 8(3), pages 618-627.
    15. Yi Bu & Binglu Wang & Win-bin Huang & Shangkun Che & Yong Huang, 2018. "Using the appearance of citations in full text on author co-citation analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(1), pages 275-289, July.
    16. Rahman, Mohammad Tariqur & Regenstein, Joe Mac & Kassim, Noor Lide Abu & Haque, Nazmul, 2017. "The need to quantify authors’ relative intellectual contributions in a multi-author paper," Journal of Informetrics, Elsevier, vol. 11(1), pages 275-281.
    17. Ana Teresa Santos & Sandro Mendonça, 2022. "Do papers (really) match journals’ “aims and scope”? A computational assessment of innovation studies," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 7449-7470, December.
    18. Lorna Wildgaard & Jesper W. Schneider & Birger Larsen, 2014. "A review of the characteristics of 108 author-level bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 125-158, October.
    19. Bar-Ilan, Judit, 2008. "Informetrics at the beginning of the 21st century—A review," Journal of Informetrics, Elsevier, vol. 2(1), pages 1-52.
    20. Xing, Yanmeng & Wang, Fenghua & Zeng, An & Ying, Fan, 2021. "Solving the cold-start problem in scientific credit allocation," Journal of Informetrics, Elsevier, vol. 15(3).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:15:y:2021:i:4:s1751157721000729. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.