IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v6y2012i4p470-479.html

Revisiting the scaling of citations for research assessment

Author

Listed:
  • Abramo, Giovanni
  • Cicero, Tindaro
  • D’Angelo, Ciriaco Andrea

Abstract

Over the past decade, national research evaluation exercises, traditionally conducted using the peer review method, have begun opening to bibliometric indicators. The citations received by a publication are assumed as proxy for its quality, but they require standardization prior to use in comparative evaluation of organizations or individual scientists: the citation data must be standardized, due to the varying citation behavior across research fields. The objective of this paper is to compare the effectiveness of the different methods of normalizing citations, in order to provide useful indications to research assessment practitioners. Simulating a typical national research assessment exercise, he analysis is conducted for all subject categories in the hard sciences and is based on the Thomson Reuters Science Citation Index-Expanded®. Comparisons show that the citations average is the most effective scaling parameter, when the average is based only on the publications actually cited.

Suggested Citation

  • Abramo, Giovanni & Cicero, Tindaro & D’Angelo, Ciriaco Andrea, 2012. "Revisiting the scaling of citations for research assessment," Journal of Informetrics, Elsevier, vol. 6(4), pages 470-479.
    • Handle: RePEc:eee:infome:v:6:y:2012:i:4:p:470-479
    DOI: 10.1016/j.joi.2012.03.005
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157712000272
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.joi.2012.03.005?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Giovanni Abramo & Ciriaco Andrea D’Angelo, 2011. "Evaluating research: from informed peer review to bibliometrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(3), pages 499-514, June.
    2. Lundberg, Jonas, 2007. "Lifting the crown—citation z-score," Journal of Informetrics, Elsevier, vol. 1(2), pages 145-154.
    3. Vieira, E.S. & Gomes, J.A.N.F., 2010. "Citations to scientific articles: Its distribution and dependence on the article features," Journal of Informetrics, Elsevier, vol. 4(1), pages 1-13.
    4. Giovanni Abramo & Ciriaco Andrea D’Angelo & Flavia Di Costa, 2011. "A national-scale cross-time analysis of university research performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(2), pages 399-413, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Abramo, Giovanni & Cicero, Tindaro & D’Angelo, Ciriaco Andrea, 2012. "How important is choice of the scaling factor in standardizing citations?," Journal of Informetrics, Elsevier, vol. 6(4), pages 645-654.
    2. Zhihui Zhang & Ying Cheng & Nian Cai Liu, 2015. "Improving the normalization effect of mean-based method from the perspective of optimization: optimization-based linear methods and their performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 587-607, January.
    3. Abramo, Giovanni & Cicero, Tindaro & D’Angelo, Ciriaco Andrea, 2012. "A sensitivity analysis of research institutions’ productivity rankings to the time of citation observation," Journal of Informetrics, Elsevier, vol. 6(2), pages 298-306.
    4. Kayvan Kousha & Mike Thelwall, 2024. "Factors associating with or predicting more cited or higher quality journal articles: An Annual Review of Information Science and Technology (ARIST) paper," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 75(3), pages 215-244, March.
    5. Abramo, Giovanni & Cicero, Tindaro & D’Angelo, Ciriaco Andrea, 2011. "A field-standardized application of DEA to national-scale research assessment of universities," Journal of Informetrics, Elsevier, vol. 5(4), pages 618-628.
    6. Giovanni Abramo & Ciriaco Andrea D’Angelo & Flavia Di Costa, 2011. "National research assessment exercises: a comparison of peer review and bibliometrics rankings," Scientometrics, Springer;Akadémiai Kiadó, vol. 89(3), pages 929-941, December.
    7. Abramo, Giovanni & Cicero, Tindaro & D’Angelo, Ciriaco Andrea, 2012. "A sensitivity analysis of researchers’ productivity rankings to the time of citation observation," Journal of Informetrics, Elsevier, vol. 6(2), pages 192-201.
    8. Schneider, Jesper W., 2013. "Caveats for using statistical significance tests in research assessments," Journal of Informetrics, Elsevier, vol. 7(1), pages 50-62.
    9. Vinkler, Péter, 2013. "Comparative rank assessment of journal articles," Journal of Informetrics, Elsevier, vol. 7(3), pages 712-717.
    10. Sangwal, Keshra, 2014. "Distributions of citations of papers of individual authors publishing in different scientific disciplines: Application of Langmuir-type function," Journal of Informetrics, Elsevier, vol. 8(4), pages 972-984.
    11. Ma, Chao & Li, Yiwei & Guo, Feng & Si, Kao, 2019. "The citation trap: Papers published at year-end receive systematically fewer citations," Journal of Economic Behavior & Organization, Elsevier, vol. 166(C), pages 667-687.
    12. Kaur, Jasleen & Radicchi, Filippo & Menczer, Filippo, 2013. "Universality of scholarly impact metrics," Journal of Informetrics, Elsevier, vol. 7(4), pages 924-932.
    13. Bornmann, Lutz & Leydesdorff, Loet & Wang, Jian, 2014. "How to improve the prediction based on citation impact percentiles for years shortly after the publication date?," Journal of Informetrics, Elsevier, vol. 8(1), pages 175-180.
    14. Patrick Herron & Aashish Mehta & Cong Cao & Timothy Lenoir, 2016. "Research diversification and impact: the case of national nanoscience development," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 629-659, November.
    15. Dunaiski, Marcel & Geldenhuys, Jaco & Visser, Willem, 2019. "On the interplay between normalisation, bias, and performance of paper impact metrics," Journal of Informetrics, Elsevier, vol. 13(1), pages 270-290.
    16. Alberto Anfossi & Alberto Ciolfi & Filippo Costa & Giorgio Parisi & Sergio Benedetto, 2016. "Large-scale assessment of research outputs through a weighted combination of bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(2), pages 671-683, May.
    17. Giovanni Abramo & Ciriaco Andrea D’Angelo & Tindaro Cicero, 2012. "What is the appropriate length of the publication period over which to assess research performance?," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(3), pages 1005-1017, December.
    18. Kong, Ling & Wang, Dongbo, 2020. "Comparison of citations and attention of cover and non-cover papers," Journal of Informetrics, Elsevier, vol. 14(4).
    19. Olugbenga Oladinrin & Kasun Gomis & Wadu Mesthrige Jayantha & Lovelin Obi & Muhammad Qasim Rana, 2021. "Scientometric Analysis of Global Scientific Literature on Aging in Place," IJERPH, MDPI, vol. 18(23), pages 1-16, November.
    20. Mehdi Rhaiem & Nabil Amara, 2020. "Determinants of research efficiency in Canadian business schools: evidence from scholar-level data," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 53-99, October.

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:6:y:2012:i:4:p:470-479. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.