IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v91y2012i3d10.1007_s11192-012-0630-z.html
   My bibliography  Save this article

Using complex networks concepts to assess approaches for citations in scientific papers

Author

Listed:
  • D. R. Amancio

    (University of São Paulo)

  • M. G. V. Nunes

    (University of São Paulo)

  • O. N. Oliveira

    (University of São Paulo)

  • L. F. Costa

    (University of São Paulo)

Abstract

The number of citations received by authors in scientific journals has become a major parameter to assess individual researchers and the journals themselves through the impact factor. A fair assessment therefore requires that the criteria for selecting references in a given manuscript should be unbiased with regard to the authors or journals cited. In this paper, we assess approaches for citations considering two recommendations for authors to follow while preparing a manuscript: (i) consider similarity of contents with the topics investigated, lest related work should be reproduced or ignored; (ii) perform a systematic search over the network of citations including seminal or very related papers. We use formalisms of complex networks for two datasets of papers from the arXiv and the Web of Science repositories to show that neither of these two criteria is fulfilled in practice. By representing the texts as complex networks we estimated a similarity index between pieces of texts and found that the list of references did not contain the most similar papers in the dataset. This was quantified by calculating a consistency index, whose maximum value is one if the references in a given paper are the most similar in the dataset. For the areas of “complex networks” and “graphenes”, the consistency index was only 0.11–0.23 and 0.10–0.25, respectively. To simulate a systematic search in the citation network, we employed a traditional random walk search (i.e. diffusion) and a random walk whose probabilities of transition are proportional to the number of the ingoing edges of the neighbours. The frequency of visits to the nodes (papers) in the network had a very small correlation with either the actual list of references in the papers or with the number of downloads from the arXiv repository. Therefore, apparently the authors and users of the repository did not follow the criterion related to a systematic search over the network of citations. Based on these results, we propose an approach that we believe is fairer for evaluating and complementing citations of a given author, effectively leading to a virtual scientometry.

Suggested Citation

  • D. R. Amancio & M. G. V. Nunes & O. N. Oliveira & L. F. Costa, 2012. "Using complex networks concepts to assess approaches for citations in scientific papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(3), pages 827-842, June.
    • Handle: RePEc:spr:scient:v:91:y:2012:i:3:d:10.1007_s11192-012-0630-z
    DOI: 10.1007/s11192-012-0630-z
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-012-0630-z
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-012-0630-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. S. Redner, 1998. "How popular is your paper? An empirical study of the citation distribution," The European Physical Journal B: Condensed Matter and Complex Systems, Springer;EDP Sciences, vol. 4(2), pages 131-134, July.
    2. Wang, Mingyang & Yu, Guang & Yu, Daren, 2009. "Effect of the age of papers on the preferential attachment in citation networks," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 388(19), pages 4273-4276.
    3. Waister Silva Martins & Marcos André Gonçalves & Alberto H. F. Laender & Nivio Ziviani, 2010. "Assessing the quality of scientific conferences based on bibliographic citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 83(1), pages 133-155, April.
    4. Amancio, D.R. & Nunes, M.G.V. & Oliveira, O.N. & Pardo, T.A.S. & Antiqueira, L. & da F. Costa, L., 2011. "Using metrics from complex networks to evaluate machine translation," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 390(1), pages 131-142.
    5. Howard D. White, 2001. "Authors as citers over time," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 52(2), pages 87-108.
    6. Hajra, Kamalika Basu & Sen, Parongama, 2005. "Aging in citation networks," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 346(1), pages 44-48.
    7. M. H. MacRoberts & B. R. MacRoberts, 1997. "Citation content analysis of a botany journal," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 48(3), pages 274-275, March.
    8. Antiqueira, L. & Nunes, M.G.V. & Oliveira Jr., O.N. & F. Costa, L. da, 2007. "Strong correlations between text quality and complex networks features," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 373(C), pages 811-820.
    9. Wright, Malcolm & Armstrong, J. Scott, 2007. "Verification of Citations: Fawlty Towers of Knowledge?," MPRA Paper 4149, University Library of Munich, Germany.
    10. Malcolm Wright & J. Scott Armstrong, 2008. "The Ombudsman: Verification of Citations: Fawlty Towers of Knowledge?," Interfaces, INFORMS, vol. 38(2), pages 125-139, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Diego R. Amancio & Osvaldo N. Oliveira jr & Luciano F. Costa, 2015. "Topological-collaborative approach for disambiguating authors’ names in collaborative networks," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 465-485, January.
    2. Silva, Filipi N. & Amancio, Diego R. & Bardosova, Maria & Costa, Luciano da F. & Oliveira, Osvaldo N., 2016. "Using network science and text analytics to produce surveys in a scientific topic," Journal of Informetrics, Elsevier, vol. 10(2), pages 487-502.
    3. Adilson Vital & Diego R. Amancio, 2022. "A comparative analysis of local similarity metrics and machine learning approaches: application to link prediction in author citation networks," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(10), pages 6011-6028, October.
    4. Corrêa Jr., Edilson A. & Silva, Filipi N. & da F. Costa, Luciano & Amancio, Diego R., 2017. "Patterns of authors contribution in scientific manuscripts," Journal of Informetrics, Elsevier, vol. 11(2), pages 498-510.
    5. Samuel Zanferdini Oliva & Livia Oliveira-Ciabati & Denise Gazotto Dezembro & Mário Sérgio Adolfi Júnior & Maísa Carvalho Silva & Hugo Cesar Pessotti & Juliana Tarossi Pollettini, 2021. "Text structuring methods based on complex network: a systematic review," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1471-1493, February.
    6. Viana, Matheus P. & Amancio, Diego R. & da F. Costa, Luciano, 2013. "On time-varying collaboration networks," Journal of Informetrics, Elsevier, vol. 7(2), pages 371-378.
    7. Xiomara S. Q. Chacon & Thiago C. Silva & Diego R. Amancio, 2020. "Comparing the impact of subfields in scientific journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 625-639, October.
    8. Amancio, Diego Raphael & Oliveira, Osvaldo Novais & da Fontoura Costa, Luciano, 2012. "Three-feature model to reproduce the topology of citation networks and the effects from authors’ visibility on their h-index," Journal of Informetrics, Elsevier, vol. 6(3), pages 427-434.
    9. Danhao Zhu & Dongbo Wang & Saeed-Ul Hassan & Peter Haddawy, 2013. "Small-world phenomenon of keywords network based on complex network," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(2), pages 435-442, November.
    10. Zhao, Qihang & Feng, Xiaodong, 2022. "Utilizing citation network structure to predict paper citation counts: A Deep learning approach," Journal of Informetrics, Elsevier, vol. 16(1).
    11. Kavitha Karunan & Hiran H. Lathabai & Thara Prabhakaran, 2017. "Discovering interdisciplinary interactions between two research fields using citation networks," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 335-367, October.
    12. Stefano Mammola & Diego Fontaneto & Alejandro Martínez & Filipe Chichorro, 2021. "Impact of the reference list features on the number of citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 785-799, January.
    13. Guillermo Armando Ronda-Pupo & J. Sylvan Katz, 2017. "The scaling relationship between degree centrality of countries and their citation-based performance on Management Information Systems," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(3), pages 1285-1299, September.
    14. Camilo Akimushkin & Diego Raphael Amancio & Osvaldo Novais Oliveira Jr., 2017. "Text Authorship Identified Using the Dynamics of Word Co-Occurrence Networks," PLOS ONE, Public Library of Science, vol. 12(1), pages 1-15, January.
    15. Henrique F. Arruda & Cesar H. Comin & Luciano da F. Costa, 2018. "How integrated are theoretical and applied physics?," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(2), pages 1113-1121, August.
    16. Akimushkin, Camilo & Amancio, Diego R. & Oliveira, Osvaldo N., 2018. "On the role of words in the network structure of texts: Application to authorship attribution," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 495(C), pages 49-58.
    17. Woon Peng Goh & Kang-Kwong Luke & Siew Ann Cheong, 2018. "Functional shortcuts in language co-occurrence networks," PLOS ONE, Public Library of Science, vol. 13(9), pages 1-18, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ren, Fu-Xin & Shen, Hua-Wei & Cheng, Xue-Qi, 2012. "Modeling the clustering in citation networks," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 391(12), pages 3533-3539.
    2. S. R. Goldberg & H. Anthony & T. S. Evans, 2015. "Modelling citation networks," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 1577-1604, December.
    3. Wu, Yan & Fu, Tom Z.J. & Chiu, Dah Ming, 2014. "Generalized preferential attachment considering aging," Journal of Informetrics, Elsevier, vol. 8(3), pages 650-658.
    4. Amancio, Diego R. & Oliveira Jr., Osvaldo N. & Costa, Luciano da F., 2012. "Structure–semantics interplay in complex networks and its effects on the predictability of similarity in texts," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 391(18), pages 4406-4419.
    5. Jiang, Xiaorui & Zhuge, Hai, 2019. "Forward search path count as an alternative indirect citation impact indicator," Journal of Informetrics, Elsevier, vol. 13(4).
    6. Liu, Yanyan & Li, Keping & Yan, Dongyang & Gu, Shuang, 2022. "A network-based CNN model to identify the hidden information in text data," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 590(C).
    7. Diego Raphael Amancio, 2015. "Comparing the topological properties of real and artificially generated scientific manuscripts," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 1763-1779, December.
    8. Amancio, Diego R. & Nunes, Maria G.V. & Oliveira, Osvaldo N. & Costa, Luciano da F., 2012. "Extractive summarization using complex networks and syntactic dependency," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 391(4), pages 1855-1864.
    9. Rabishankar Giri & Sabuj Kumar Chaudhuri, 2021. "Ranking journals through the lens of active visibility," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(3), pages 2189-2208, March.
    10. Željko Stević & Irena Đalić & Dragan Pamučar & Zdravko Nunić & Slavko Vesković & Marko Vasiljević & Ilija Tanackov, 2019. "A new hybrid model for quality assessment of scientific conferences based on Rough BWM and SERVQUAL," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(1), pages 1-30, April.
    11. Jianhua Hou, 2017. "Exploration into the evolution and historical roots of citation analysis by referenced publication year spectroscopy," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(3), pages 1437-1452, March.
    12. Wei, Daijun & Deng, Xinyang & Zhang, Xiaoge & Deng, Yong & Mahadevan, Sankaran, 2013. "Identifying influential nodes in weighted networks based on evidence theory," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 392(10), pages 2564-2575.
    13. Perc, Matjaž, 2010. "Zipf’s law and log-normal distributions in measures of scientific output across fields and institutions: 40 years of Slovenia’s research as an example," Journal of Informetrics, Elsevier, vol. 4(3), pages 358-364.
    14. Frederique Bordignon, 2020. "Self-correction of science: a comparative study of negative citations and post-publication peer review," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(2), pages 1225-1239, August.
    15. Ding, Waverly & Choi, Emily, 2008. "Divergent Paths or Stepping Stones: A Comparison of Scientists’ Advising and Founding Activities," Institute for Research on Labor and Employment, Working Paper Series qt4907j25p, Institute of Industrial Relations, UC Berkeley.
    16. He, Xuan & Zhao, Hai & Cai, Wei & Liu, Zheng & Si, Shuai-Zong, 2014. "Earthquake networks based on space–time influence domain," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 407(C), pages 175-184.
    17. Maziar Montazerian & Edgar Dutra Zanotto & Hellmut Eckert, 2019. "A new parameter for (normalized) evaluation of H-index: countries as a case study," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(3), pages 1065-1078, March.
    18. Martins, Francisco Leonardo Bezerra & do Nascimento, José Cláudio, 2022. "Power law dynamics in genealogical graphs," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 596(C).
    19. J. Martin Zyl, 2013. "The generalized Pareto distribution fitted to research outputs of countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(3), pages 1099-1109, March.
    20. Young-Ho Eom & Santo Fortunato, 2011. "Characterizing and Modeling Citation Dynamics," PLOS ONE, Public Library of Science, vol. 6(9), pages 1-7, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:91:y:2012:i:3:d:10.1007_s11192-012-0630-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.