IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v126y2021i10d10.1007_s11192-021-04115-6.html
   My bibliography  Save this article

Weighted citation based on ranking-related contribution: a new index for evaluating article impact

Author

Listed:
  • Yuanyuan Liu

    (University of Science and Technology of China)

  • Qiang Wu

    (University of Science and Technology of China)

  • Shijie Wu

    (University of Science and Technology of China)

  • Yong Gao

    (University of Science and Technology of China)

Abstract

Citation count fails to comprehensively and accurately portray the publication’s impact, and the scientific impact is far more than what we see on the surface. This paper proposes a new impact indicator, the Contribution-Weighted Citation index, in which each citation is weighted from the perspective of the citing item. The method of weighting is based on ranking of the ‘old’ impact values of articles published in the same year and same field, and modifies the value of one citation from each publication positively or negatively. The idea of iteration is introduced into the calculation process for more accurate results. We empirically analysed 3,847,243 papers from the WoS database for the 1978–2017 period, including the fields of Mathematics, Physics, and Space Science. The experimental results show that the three variants of the new index proposed in this article show different characteristics in identifying article impact. Our new indicator in sine mode make articles more discrete, which helps distinguish between papers with similar citation count, while the arcsine mode is more helpful to identify high-citation-quality papers. In addition, we found the average citation quality of lowly cited papers fluctuates greatly, and some such papers have surprisingly high citation quality; for highly cited case, the average citation quality stabilizes in a certain range. This article has a positive role in identifying important publications and under-cited publications.

Suggested Citation

  • Yuanyuan Liu & Qiang Wu & Shijie Wu & Yong Gao, 2021. "Weighted citation based on ranking-related contribution: a new index for evaluating article impact," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(10), pages 8653-8672, October.
    • Handle: RePEc:spr:scient:v:126:y:2021:i:10:d:10.1007_s11192-021-04115-6
    DOI: 10.1007/s11192-021-04115-6
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-021-04115-6
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-021-04115-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Erjia Yan & Cassidy R. Sugimoto, 2011. "Institutional interactions: Exploring social, cognitive, and geographic relationships between institutions as demonstrated through citation networks," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(8), pages 1498-1514, August.
    2. Xiaomei Bai & Feng Xia & Ivan Lee & Jun Zhang & Zhaolong Ning, 2016. "Identifying Anomalous Citations for Objective Evaluation of Scholarly Article Impact," PLOS ONE, Public Library of Science, vol. 11(9), pages 1-15, September.
    3. Ruiz-Castillo, Javier, 2013. "The comparison of classification-system-based normalization procedures with source normalization alternatives in Waltman and Van Eck (2013)," UC3M Working papers. Economics we1318, Universidad Carlos III de Madrid. Departamento de Economía.
    4. Su, Cheng & Pan, YunTao & Zhen, YanNing & Ma, Zheng & Yuan, JunPeng & Guo, Hong & Yu, ZhengLu & Ma, CaiFeng & Wu, YiShan, 2011. "PrestigeRank: A new evaluation method for papers and journals," Journal of Informetrics, Elsevier, vol. 5(1), pages 1-13.
    5. Rolf Ketzler & Klaus F. Zimmermann, 2013. "A citation-analysis of economic research institutes," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(3), pages 1095-1112, June.
    6. Björn Hammarfelt, 2011. "Citation analysis on the micro level: The example of Walter Benjamin's Illuminations," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(5), pages 819-830, May.
    7. Richard Klavans & Kevin W. Boyack, 2017. "Which Type of Citation Analysis Generates the Most Accurate Taxonomy of Scientific and Technical Knowledge?," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 68(4), pages 984-998, April.
    8. Björn Hammarfelt, 2011. "Citation analysis on the micro level: The example of Walter Benjamin's Illuminations," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(5), pages 819-830, May.
    9. Erjia Yan & Cassidy R. Sugimoto, 2011. "Institutional interactions: Exploring social, cognitive, and geographic relationships between institutions as demonstrated through citation networks," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(8), pages 1498-1514, August.
    10. Erjia Yan & Ying Ding, 2010. "Weighted citation: An indicator of an article's prestige," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 61(8), pages 1635-1643, August.
    11. András Schubert, 2009. "Using the h-index for assessing single publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 78(3), pages 559-565, March.
    12. Xiaodan Zhu & Peter Turney & Daniel Lemire & André Vellino, 2015. "Measuring academic influence: Not all citations are equal," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(2), pages 408-427, February.
    13. Chen, P. & Xie, H. & Maslov, S. & Redner, S., 2007. "Finding scientific gems with Google’s PageRank algorithm," Journal of Informetrics, Elsevier, vol. 1(1), pages 8-15.
    14. Michel Zitt & Henry Small, 2008. "Modifying the journal impact factor by fractional citation weighting: The audience factor," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(11), pages 1856-1860, September.
    15. Erjia Yan & Ying Ding & Cassidy R. Sugimoto, 2011. "P‐Rank: An indicator measuring prestige in heterogeneous scholarly networks," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(3), pages 467-477, March.
    16. Eleni Fragkiadaki & Georgios Evangelidis, 2014. "Review of the indirect citations paradigm: theory and practice of the assessment of papers, authors and journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 99(2), pages 261-288, May.
    17. Ludo Waltman & Nees Jan Eck, 2013. "Source normalized indicators of citation impact: an overview of different approaches and an empirical comparison," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(3), pages 699-716, September.
    18. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    19. Philip M. Davis, 2008. "Eigenfactor: Does the principle of repeated improvement result in better estimates than raw citation counts?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(13), pages 2186-2188, November.
    20. Egghe, L. & Bornmann, L. & Guns, R., 2011. "A proposal for a First-Citation-Speed-Index," Journal of Informetrics, Elsevier, vol. 5(1), pages 181-186.
    21. Giuffrida, Cristiano & Abramo, Giovanni & D’Angelo, Ciriaco Andrea, 2019. "Are all citations worth the same? Valuing citations by the value of the citing items," Journal of Informetrics, Elsevier, vol. 13(2), pages 500-514.
    22. Jiang, Xiaorui & Zhuge, Hai, 2019. "Forward search path count as an alternative indirect citation impact indicator," Journal of Informetrics, Elsevier, vol. 13(4).
    23. Ammar Nejati & Seyyed Mehdi Hosseini Jenab, 2010. "A two-dimensional approach to evaluate the scientific production of countries (case study: the basic sciences)," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(2), pages 357-364, August.
    24. Erjia Yan & Ying Ding, 2010. "Weighted citation: An indicator of an article's prestige," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(8), pages 1635-1643, August.
    25. L. Egghe, 2010. "On the relation between Schubert’s h-index of a single paper and its total number of received citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(1), pages 115-117, July.
    26. Hu, Xiaojun & Rousseau, Ronald & Chen, Jin, 2011. "On the definition of forward and backward citation generations," Journal of Informetrics, Elsevier, vol. 5(1), pages 27-36.
    27. Franceschet, Massimo, 2010. "The difference between popularity and prestige in the sciences and in the social sciences: A bibliometric analysis," Journal of Informetrics, Elsevier, vol. 4(1), pages 55-63.
    28. Abramo, Giovanni & Cicero, Tindaro & D’Angelo, Ciriaco Andrea, 2012. "Revisiting the scaling of citations for research assessment," Journal of Informetrics, Elsevier, vol. 6(4), pages 470-479.
    29. Erjia Yan & Ying Ding & Cassidy R. Sugimoto, 2011. "P-Rank: An indicator measuring prestige in heterogeneous scholarly networks," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(3), pages 467-477, March.
    30. Dalibor Fiala & François Rousselot & Karel Ježek, 2008. "PageRank for bibliographic networks," Scientometrics, Springer;Akadémiai Kiadó, vol. 76(1), pages 135-158, July.
    31. Lundberg, Jonas, 2007. "Lifting the crown—citation z-score," Journal of Informetrics, Elsevier, vol. 1(2), pages 145-154.
    32. Liwei Cai & Jiahao Tian & Jiaying Liu & Xiaomei Bai & Ivan Lee & Xiangjie Kong & Feng Xia, 2019. "Scholarly impact assessment: a survey of citation weighting solutions," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(2), pages 453-478, February.
    33. Vaccario, Giacomo & Medo, Matúš & Wider, Nicolas & Mariani, Manuel Sebastian, 2017. "Quantifying and suppressing ranking bias in a large citation network," Journal of Informetrics, Elsevier, vol. 11(3), pages 766-782.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Linhong Xu & Kun Ding & Yuan Lin & Chunbo Zhang, 2023. "Does citation polarity help evaluate the quality of academic papers?," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(7), pages 4065-4087, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Liwei Cai & Jiahao Tian & Jiaying Liu & Xiaomei Bai & Ivan Lee & Xiangjie Kong & Feng Xia, 2019. "Scholarly impact assessment: a survey of citation weighting solutions," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(2), pages 453-478, February.
    2. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    3. Yu Zhang & Min Wang & Morteza Saberi & Elizabeth Chang, 2022. "Analysing academic paper ranking algorithms using test data and benchmarks: an investigation," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(7), pages 4045-4074, July.
    4. Jiang, Xiaorui & Zhuge, Hai, 2019. "Forward search path count as an alternative indirect citation impact indicator," Journal of Informetrics, Elsevier, vol. 13(4).
    5. Zhang, Fang & Wu, Shengli, 2020. "Predicting future influence of papers, researchers, and venues in a dynamic academic network," Journal of Informetrics, Elsevier, vol. 14(2).
    6. Lyu, Haihua & Bu, Yi & Zhao, Zhenyue & Zhang, Jiarong & Li, Jiang, 2022. "Citation bias in measuring knowledge flow: Evidence from the web of science at the discipline level," Journal of Informetrics, Elsevier, vol. 16(4).
    7. Eleni Fragkiadaki & Georgios Evangelidis, 2016. "Three novel indirect indicators for the assessment of papers and authors based on generations of citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(2), pages 657-694, February.
    8. Fang Zhang & Shengli Wu, 2021. "Measuring academic entities’ impact by content-based citation analysis in a heterogeneous academic network," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(8), pages 7197-7222, August.
    9. Bouyssou, Denis & Marchant, Thierry, 2016. "Ranking authors using fractional counting of citations: An axiomatic approach," Journal of Informetrics, Elsevier, vol. 10(1), pages 183-199.
    10. Wang, Jingjing & Xu, Shuqi & Mariani, Manuel S. & Lü, Linyuan, 2021. "The local structure of citation networks uncovers expert-selected milestone papers," Journal of Informetrics, Elsevier, vol. 15(4).
    11. Lutz Bornmann & Alexander Tekles & Loet Leydesdorff, 2019. "How well does I3 perform for impact measurement compared to other bibliometric indicators? The convergent validity of several (field-normalized) indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(2), pages 1187-1205, May.
    12. Tehmina Amjad & Ying Ding & Ali Daud & Jian Xu & Vincent Malic, 2015. "Topic-based heterogeneous rank," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(1), pages 313-334, July.
    13. Bornmann, Lutz & Haunschild, Robin & Mutz, Rüdiger, 2020. "Should citations be field-normalized in evaluative bibliometrics? An empirical analysis based on propensity score matching," Journal of Informetrics, Elsevier, vol. 14(4).
    14. Zhou, Ping & Leydesdorff, Loet, 2011. "Fractional counting of citations in research evaluation: A cross- and interdisciplinary assessment of the Tsinghua University in Beijing," Journal of Informetrics, Elsevier, vol. 5(3), pages 360-368.
    15. Dunaiski, Marcel & Geldenhuys, Jaco & Visser, Willem, 2019. "Globalised vs averaged: Bias and ranking performance on the author level," Journal of Informetrics, Elsevier, vol. 13(1), pages 299-313.
    16. Dunaiski, Marcel & Geldenhuys, Jaco & Visser, Willem, 2019. "On the interplay between normalisation, bias, and performance of paper impact metrics," Journal of Informetrics, Elsevier, vol. 13(1), pages 270-290.
    17. Zhenbin Yan & Qiang Wu & Xingchen Li, 2016. "Do Hirsch-type indices behave the same in assessing single publications? An empirical study of 29 bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1815-1833, December.
    18. Ludo Waltman & Erjia Yan & Nees Jan Eck, 2011. "A recursive field-normalized bibliometric performance indicator: an application to the field of library and information science," Scientometrics, Springer;Akadémiai Kiadó, vol. 89(1), pages 301-314, October.
    19. Cristina López-Duarte & Marta M. Vidal-Suárez & Belén González-Díaz, 2019. "Cross-national distance and international business: an analysis of the most influential recent models," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(1), pages 173-208, October.
    20. Mingers, John & Leydesdorff, Loet, 2015. "A review of theory and practice in scientometrics," European Journal of Operational Research, Elsevier, vol. 246(1), pages 1-19.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:126:y:2021:i:10:d:10.1007_s11192-021-04115-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.