IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v14y2020i1s1751157719302640.html
   My bibliography  Save this article

Self-Citations and scientific evaluation: Leadership, influence, and performance

Author

Listed:
  • Simoes, Nadia
  • Crespo, Nuno

Abstract

The hα index was recently proposed by Hirsch (2019) to measure the degree of scientific leadership. However, as discussed in recent literature, this measure has important shortcomings. We introduce an alternative approach that does not suffer from these limitations. It uses self-citations as a source of information, makes the evaluation at the paper-level, and centers the analysis on the new and broader concept of scientific influence. In each specific paper, the level of scientific influence of an author ranges between 0 and 1 and corresponds to his/her share in the total number of self-citations in that paper. Moreover, we show how this concept can be used to produce a more accurate co-authorship weighting scheme that allows for adjusting the standard measures of scientific performance. This is particularly important in the case of areas in which the alphabetical order of the names is the rule commonly followed, since it provides a way to differentiate the authors according to their role in each paper. We illustrate our method with an empirical example, comparing male and female economists of world top universities. The evidence highlights the existence of substantial gender differences in terms of scientific leadership and scientific influence.

Suggested Citation

  • Simoes, Nadia & Crespo, Nuno, 2020. "Self-Citations and scientific evaluation: Leadership, influence, and performance," Journal of Informetrics, Elsevier, vol. 14(1).
    • Handle: RePEc:eee:infome:v:14:y:2020:i:1:s1751157719302640
    DOI: 10.1016/j.joi.2019.100990
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157719302640
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.joi.2019.100990?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Beaudry, Catherine & Larivière, Vincent, 2016. "Which gender gap? Factors affecting researchers’ scientific impact in science and medicine," Research Policy, Elsevier, vol. 45(9), pages 1790-1817.
    2. Yannick Berker, 2018. "Golden-ratio as a substitute to geometric and harmonic counting to determine multi-author publication credit," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 839-857, March.
    3. J. E. Hirsch, 2010. "An index to quantify an individual’s scientific research output that takes into account the effect of multiple coauthorship," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(3), pages 741-754, December.
    4. P. Pichappan & S. Sarasvady, 2002. "The other side of the coin: The intricacies of author self-citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 54(2), pages 285-290, June.
    5. Seeber, Marco & Cattaneo, Mattia & Meoli, Michele & Malighetti, Paolo, 2019. "Self-citations as strategic response to the use of metrics for career decisions," Research Policy, Elsevier, vol. 48(2), pages 478-491.
    6. Dangzhi Zhao & Andreas Strotmann & Alicia Cappello, 2018. "In†text function of author self†citations: Implications for research evaluation practice," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 69(7), pages 949-952, July.
    7. Ash Mohammad Abbas, 2011. "Weighted indices for evaluating the quality of research with multiple authorship," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(1), pages 107-131, July.
    8. Rahman, Mohammad Tariqur & Regenstein, Joe Mac & Kassim, Noor Lide Abu & Haque, Nazmul, 2017. "The need to quantify authors’ relative intellectual contributions in a multi-author paper," Journal of Informetrics, Elsevier, vol. 11(1), pages 275-281.
    9. Maxim Engers & Joshua S. Gans & Simon Grant & Stephen King, 1999. "First-Author Conditions," Journal of Political Economy, University of Chicago Press, vol. 107(4), pages 859-883, August.
    10. João M. Fernandes & Miguel P. Monteiro, 2017. "Evolution in the number of authors of computer science publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(2), pages 529-539, February.
    11. Maria De Paola & Michela Ponzo & Vincenzo Scoppa, 2018. "Are Men Given Priority for Top Jobs? Investigating the Glass Ceiling in Italian Academia," Journal of Human Capital, University of Chicago Press, vol. 12(3), pages 475-503.
    12. Henk F Moed, 2007. "The future of research evaluation rests with an intelligent combination of advanced metrics and transparent peer review," Science and Public Policy, Oxford University Press, vol. 34(8), pages 575-583, October.
    13. Kosmulski, Marek, 2012. "The order in the lists of authors in multi-author papers revisited," Journal of Informetrics, Elsevier, vol. 6(4), pages 639-644.
    14. Eduardo B Araújo & Nuno A M Araújo & André A Moreira & Hans J Herrmann & José S Andrade Jr., 2017. "Gender differences in scientific collaborations: Women are more egalitarian than men," PLOS ONE, Public Library of Science, vol. 12(5), pages 1-10, May.
    15. Pablo Dorta-González & María-Isabel Dorta-González, 2011. "Central indexes to the citation distribution: a complement to the h-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(3), pages 729-745, September.
    16. Lorna Wildgaard & Jesper W. Schneider & Birger Larsen, 2014. "A review of the characteristics of 108 author-level bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 125-158, October.
    17. Daniel S. Hamermesh, 2018. "Citations in Economics: Measurement, Uses, and Impacts," Journal of Economic Literature, American Economic Association, vol. 56(1), pages 115-156, March.
    18. Sabrina J. Mayer & Justus M. K. Rathmann, 2018. "How does research productivity relate to gender? Analyzing gender differences for multiple publication dimensions," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(3), pages 1663-1693, December.
    19. Anthony F. J. Raan, 2006. "Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups," Scientometrics, Springer;Akadémiai Kiadó, vol. 67(3), pages 491-502, June.
    20. Efthyvoulou, Georgios, 2008. "Alphabet Economics: The link between names and reputation," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 37(3), pages 1266-1285, June.
    21. I. Lukovits & P. Vinkler, 1995. "Correct credit distribution: A model for sharing credit among coauthors," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 36(1), pages 91-98, September.
    22. Crespo, Nuno & Simoes, Nadia, 2019. "Publication Performance and Number of Authors – Evidence for World Top Economists," MPRA Paper 92581, University Library of Munich, Germany.
    23. J. E. Hirsch, 2019. "hα: An index to quantify an individual’s scientific leadership," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(2), pages 673-686, February.
    24. Bozeman, Barry & Gaughan, Monica, 2011. "How do men and women differ in research collaborations? An analysis of the collaborative motives and strategies of academic researchers," Research Policy, Elsevier, vol. 40(10), pages 1393-1402.
    25. David N. Laband & Robert D. Tollison, 2000. "Intellectual Collaboration," Journal of Political Economy, University of Chicago Press, vol. 108(3), pages 632-661, June.
    26. Leo Egghe, 2006. "Theory and practise of the g-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 69(1), pages 131-152, October.
    27. Xiaojun Hu, 2009. "Loads of special authorship functions: Linear growth in the percentage of “equal first authors” and corresponding authors," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(11), pages 2378-2381, November.
    28. Brooks, Chris & Fenton, Evelyn M. & Walker, James T., 2014. "Gender and the evaluation of research," Research Policy, Elsevier, vol. 43(6), pages 990-1001.
    29. Lisa Geraci & Steve Balsis & Alexander J. Busch Busch, 2015. "Gender and the h index in psychology," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 2023-2034, December.
    30. Anna Tietze & Philip Hofmann, 2019. "The h-index and multi-author hm-index for individual researchers in condensed matter physics," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(1), pages 171-185, April.
    31. Paul J. Boyle & Lucy K. Smith & Nicola J. Cooper & Kate S. Williams & Henrietta O'Connor, 2015. "Gender balance: Women are funded more fairly in social science," Nature, Nature, vol. 525(7568), pages 181-183, September.
    32. Kadel, Annke & Walter, Andreas, 2015. "Do scholars in Economics and Finance react to alphabetical discrimination?," Finance Research Letters, Elsevier, vol. 14(C), pages 64-68.
    33. Leo Egghe & Raf Guns & Ronald Rousseau, 2013. "Measuring co-authors’ contribution to an article’s visibility," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(1), pages 55-67, April.
    34. Loet Leydesdorff & Lutz Bornmann & Tobias Opthof, 2019. "hα: the scientist as chimpanzee or bonobo," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(3), pages 1163-1166, March.
    35. Frandsen, Tove Faber & Nicolaisen, Jeppe, 2010. "What is in a name? Credit assignment practices in different disciplines," Journal of Informetrics, Elsevier, vol. 4(4), pages 608-617.
    36. Björn Hammarfelt & Alexander D. Rushforth, 2017. "Indicators as judgment devices: An empirical study of citizen bibliometrics in research evaluation," Research Evaluation, Oxford University Press, vol. 26(3), pages 169-180.
    37. Vincent Larivière & Yves Gingras & Cassidy R. Sugimoto & Andrew Tsou, 2015. "Team size matters: Collaboration and scientific impact since 1900," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(7), pages 1323-1332, July.
    38. António Osório, 2018. "On the impossibility of a perfect counting method to allocate the credits of multi-authored publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 2161-2173, September.
    39. Abramo, Giovanni & D’Angelo, Ciriaco Andrea & Rosati, Francesco, 2013. "The importance of accounting for the number of co-authors and their order when assessing research performance at the individual level in the life sciences," Journal of Informetrics, Elsevier, vol. 7(1), pages 198-208.
    40. Lukas Kuld & John O’Hagan, 2018. "Rise of multi-authored papers in economics: Demise of the ‘lone star’ and why?," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 1207-1225, March.
    41. Teja Tscharntke & Michael E Hochberg & Tatyana A Rand & Vincent H Resh & Jochen Krauss, 2007. "Author Sequence and Credit for Contributions in Multiauthored Publications," PLOS Biology, Public Library of Science, vol. 5(1), pages 1-2, January.
    42. Leo Egghe, 2008. "Mathematical theory of the h‐ and g‐index in case of fractional counting of authorship," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(10), pages 1608-1616, August.
    43. Liu, Xuan Zhen & Fang, Hui, 2012. "Modifying h-index by allocating credit of multi-authored papers whose author names rank based on contribution," Journal of Informetrics, Elsevier, vol. 6(4), pages 557-565.
    44. Bornmann, Lutz & Osório, António, 2019. "The value and credits of n-authors publications," Journal of Informetrics, Elsevier, vol. 13(2), pages 540-554.
    45. Wolfgang Glänzel & Koenraad Debackere & Bart Thijs & András Schubert, 2006. "A concise review on the role of author self-citations in information science, bibliometrics and science policy," Scientometrics, Springer;Akadémiai Kiadó, vol. 67(2), pages 263-277, May.
    46. Goodall, Amanda H., 2012. "A Theory of Expert Leadership," IZA Discussion Papers 6566, Institute of Labor Economics (IZA).
    47. Lazear, Edward P., 2012. "Leadership: A personnel economics approach," Labour Economics, Elsevier, vol. 19(1), pages 92-101.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mansour Haghighat & Javad Hayatdavoudi, 2021. "How hot are hot papers? The issue of prolificacy and self-citation stacking," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 565-578, January.
    2. Ana Teresa Santos & Sandro Mendonça, 2022. "The small world of innovation studies: an “editormetrics” perspective," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 7471-7486, December.
    3. Martin Szomszor & David A. Pendlebury & Jonathan Adams, 2020. "How much is too much? The difference between research influence and self-citation excess," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(2), pages 1119-1147, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    2. Nadia Simoes & Nuno Crespo, 2020. "A flexible approach for measuring author-level publishing performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(1), pages 331-355, January.
    3. Crespo, Nuno & Simoes, Nadia, 2019. "Publication Performance and Number of Authors – Evidence for World Top Economists," MPRA Paper 92581, University Library of Munich, Germany.
    4. Osório, António (António Miguel), 2019. "The value and credits of n-authors publications," Working Papers 2072/376026, Universitat Rovira i Virgili, Department of Economics.
    5. Waltman, Ludo, 2012. "An empirical analysis of the use of alphabetical authorship in scientific publishing," Journal of Informetrics, Elsevier, vol. 6(4), pages 700-711.
    6. Jingda Ding & Chao Liu & Qiao Zheng & Wei Cai, 2021. "A new method of co-author credit allocation based on contributor roles taxonomy: proof of concept and evaluation using papers published in PLOS ONE," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 7561-7581, September.
    7. Lathabai, Hiran H., 2020. "ψ-index: A new overall productivity index for actors of science and technology," Journal of Informetrics, Elsevier, vol. 14(4).
    8. Rahman, Mohammad Tariqur & Regenstein, Joe Mac & Kassim, Noor Lide Abu & Haque, Nazmul, 2017. "The need to quantify authors’ relative intellectual contributions in a multi-author paper," Journal of Informetrics, Elsevier, vol. 11(1), pages 275-281.
    9. Siying Li & Huawei Shen & Peng Bao & Xueqi Cheng, 2021. "$$h_u$$ h u -index: a unified index to quantify individuals across disciplines," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(4), pages 3209-3226, April.
    10. Deming Lin & Tianhui Gong & Wenbin Liu & Martin Meyer, 2020. "An entropy-based measure for the evolution of h index research," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2283-2298, December.
    11. Matthias Weber, 2016. "The Effects of Listing Authors in Alphabetical Order: A survey of the Empirical Evidence," Bank of Lithuania Occasional Paper Series 12, Bank of Lithuania.
    12. Kim, Jinseok & Kim, Jinmo, 2015. "Rethinking the comparison of coauthorship credit allocation schemes," Journal of Informetrics, Elsevier, vol. 9(3), pages 667-673.
    13. J. E. Hirsch, 2019. "hα: An index to quantify an individual’s scientific leadership," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(2), pages 673-686, February.
    14. António Osório, 2018. "On the impossibility of a perfect counting method to allocate the credits of multi-authored publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 2161-2173, September.
    15. Pär Sundling, 2023. "Author contributions and allocation of authorship credit: testing the validity of different counting methods in the field of chemical biology," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(5), pages 2737-2762, May.
    16. Martin Szomszor & David A. Pendlebury & Jonathan Adams, 2020. "How much is too much? The difference between research influence and self-citation excess," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(2), pages 1119-1147, May.
    17. Rodrigo Costas & María Bordons, 2011. "Do age and professional rank influence the order of authorship in scientific publications? Some evidence from a micro-level perspective," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(1), pages 145-161, July.
    18. Zhai, Li & Yan, Xiangbin, 2022. "A directed collaboration network for exploring the order of scientific collaboration," Journal of Informetrics, Elsevier, vol. 16(4).
    19. Xuan Zhen Liu & Hui Fang, 2014. "Scientific group leaders’ authorship preferences: an empirical investigation," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(2), pages 909-925, February.
    20. Du Jian & Tang Xiaoli, 2013. "Perceptions of author order versus contribution among researchers with different professional ranks and the potential of harmonic counts for encouraging ethical co-authorship practices," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(1), pages 277-295, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:14:y:2020:i:1:s1751157719302640. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.