IDEAS home Printed from https://ideas.repec.org/p/aeg/report/2012-03.html
   My bibliography  Save this paper

An Empirical Approach to Compare the Performance of Heterogeneous Academic Fields

Author

Listed:
  • Cinzia Daraio

    (Department of Computer, Control and Management Engineering, Universita' degli Studi di Roma "La Sapienza")

  • Giancarlo Ruocco

    (Department of Computer, Control and Management Engineering, Universita' degli Studi di Roma "La Sapienza")

Abstract

In this paper we propose a ìscaling-basedî empirical approach to assess the scientific performance of heterogeneous academic disciplines. It relies on the idea that if we take into account for their two main sources of heterogeneity, the bibliometric distributions of different academic fields can be superimposed and collapse to a unique master curve by a single scaling parameter. By using data on the scientific production of around 2,500 scholars of the university of Rome ìLa Sapienzaî from the Web of Science (WoS) over 2004ñ2008 we i) demonstrate the existence of a master curve; ii) determine the scaling factors which are the cornerstone to compare different academic fields; and iii) show that the master bibliometric distribution follows a Log-normal law.

Suggested Citation

  • Cinzia Daraio & Giancarlo Ruocco, 2012. "An Empirical Approach to Compare the Performance of Heterogeneous Academic Fields," DIAG Technical Reports 2012-03, Department of Computer, Control and Management Engineering, Universita' degli Studi di Roma "La Sapienza".
    • Handle: RePEc:aeg:report:2012-03
    as

    Download full text from publisher

    File URL: http://www.dis.uniroma1.it/~bibdis/RePEc/aeg/report/2012-03.pdf
    File Function: First version, 2012
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. T. S. Evans & N. Hopkins & B. S. Kaube, 2012. "Universality of performance indicators based on citation and reference counts," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(2), pages 473-495, November.
    2. Rodrigo Costas & Thed N. van Leeuwen & María Bordons, 2010. "A bibliometric classificatory approach for the study and assessment of research performance at the individual level: The effects of age on productivity and impact," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 61(8), pages 1564-1581, August.
    3. Pedro Albarrán & Juan A. Crespo & Ignacio Ortuño & Javier Ruiz-Castillo, 2011. "The skewness of science in 219 sub-fields and a number of aggregates," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(2), pages 385-397, August.
    4. Rodrigo Costas & Thed N. van Leeuwen & María Bordons, 2010. "A bibliometric classificatory approach for the study and assessment of research performance at the individual level: The effects of age on productivity and impact," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(8), pages 1564-1581, August.
    5. Herman Van den Berghe & Josee A. Houben & Renger E. de Bruin & Henk F. Moed & André Kint & Marc Luwel & Eric H. J. Spruyt, 1998. "Bibliometric indicators of university research performance in Flanders," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 49(1), pages 59-67.
    6. Enrico Deiaco & Magnus Holmén & Maureen McKelvey, 2009. "What Does it Mean Conceptually that Universities Compete?," Chapters, in: Maureen McKelvey & Magnus Holmén (ed.), Learning to Compete in European Universities, chapter 12, Edward Elgar Publishing.
    7. S. Redner, 1998. "How popular is your paper? An empirical study of the citation distribution," The European Physical Journal B: Condensed Matter and Complex Systems, Springer;EDP Sciences, vol. 4(2), pages 131-134, July.
    8. Daraio, Cinzia & Moed, Henk F., 2011. "Is Italian science declining?," Research Policy, Elsevier, vol. 40(10), pages 1380-1392.
    9. Wolfgang G. Stock, 2006. "On relevance distributions," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 57(8), pages 1126-1129, June.
    10. Daraio, Cinzia & Bonaccorsi, Andrea & Geuna, Aldo & Lepori, Benedetto & Bach, Laurent & Bogetoft, Peter & F. Cardoso, Margarida & Castro-Martinez, Elena & Crespi, Gustavo & de Lucio, Ignacio Fernandez, 2011. "The European university landscape: A micro characterization based on evidence from the Aquameth project," Research Policy, Elsevier, vol. 40(1), pages 148-164, February.
    11. Derek De Solla Price, 1976. "A general theory of bibliometric and other cumulative advantage processes," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 27(5), pages 292-306, September.
    12. Enrico Deiaco & Alan Hughes & Maureen McKelvey, 2012. "Universities as strategic actors in the knowledge economy," Cambridge Journal of Economics, Cambridge Political Economy Society, vol. 36(3), pages 525-541.
    13. Anthony F. J. Raan, 2006. "Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups," Scientometrics, Springer;Akadémiai Kiadó, vol. 67(3), pages 491-502, June.
    14. Per O. Seglen, 1992. "The skewness of science," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 43(9), pages 628-638, October.
    15. Wolfgang Glänzel, 2010. "The role of the h-index and the characteristic scores and scales in testing the tail properties of scientometric distributions," Scientometrics, Springer;Akadémiai Kiadó, vol. 83(3), pages 697-709, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Giancarlo Ruocco & Cinzia Daraio, 2013. "An empirical approach to compare the performance of heterogeneous academic fields," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(3), pages 601-625, December.
    2. Vîiu, Gabriel-Alexandru, 2017. "Disaggregated research evaluation through median-based characteristic scores and scales: a comparison with the mean-based approach," Journal of Informetrics, Elsevier, vol. 11(3), pages 748-765.
    3. S. R. Goldberg & H. Anthony & T. S. Evans, 2015. "Modelling citation networks," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 1577-1604, December.
    4. Ruiz-Castillo, Javier & Costas, Rodrigo, 2014. "The skewness of scientific productivity," Journal of Informetrics, Elsevier, vol. 8(4), pages 917-934.
    5. Thelwall, Mike, 2016. "The discretised lognormal and hooked power law distributions for complete citation data: Best options for modelling and regression," Journal of Informetrics, Elsevier, vol. 10(2), pages 336-346.
    6. Giovanni Abramo & Ciriaco Andrea D’Angelo, 2011. "National-scale research performance assessment at the individual level," Scientometrics, Springer;Akadémiai Kiadó, vol. 86(2), pages 347-364, February.
    7. Pedro Albarrán & Juan A. Crespo & Ignacio Ortuño & Javier Ruiz-Castillo, 2011. "The skewness of science in 219 sub-fields and a number of aggregates," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(2), pages 385-397, August.
    8. Andrea Bonaccorsi & Cinzia Daraio & Stefano Fantoni & Viola Folli & Marco Leonetti & Giancarlo Ruocco, 2017. "Do social sciences and humanities behave like life and hard sciences?," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(1), pages 607-653, July.
    9. Zhihui Zhang & Ying Cheng & Nian Cai Liu, 2015. "Improving the normalization effect of mean-based method from the perspective of optimization: optimization-based linear methods and their performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 587-607, January.
    10. Stegehuis, Clara & Litvak, Nelly & Waltman, Ludo, 2015. "Predicting the long-term citation impact of recent publications," Journal of Informetrics, Elsevier, vol. 9(3), pages 642-657.
    11. Vîiu, Gabriel-Alexandru, 2018. "The lognormal distribution explains the remarkable pattern documented by characteristic scores and scales in scientometrics," Journal of Informetrics, Elsevier, vol. 12(2), pages 401-415.
    12. Michal Brzezinski, 2015. "Power laws in citation distributions: evidence from Scopus," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(1), pages 213-228, April.
    13. Andersen, Jens Peter & Nielsen, Mathias Wullum, 2018. "Google Scholar and Web of Science: Examining gender differences in citation coverage across five scientific disciplines," Journal of Informetrics, Elsevier, vol. 12(3), pages 950-959.
    14. Andersen, Jens Peter, 2017. "An empirical and theoretical critique of the Euclidean index," Journal of Informetrics, Elsevier, vol. 11(2), pages 455-465.
    15. Brito, Ricardo & Rodríguez-Navarro, Alonso, 2018. "Research assessment by percentile-based double rank analysis," Journal of Informetrics, Elsevier, vol. 12(1), pages 315-329.
    16. Alonso Rodríguez-Navarro & Ricardo Brito, 2019. "Probability and expected frequency of breakthroughs: basis and use of a robust method of research assessment," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(1), pages 213-235, April.
    17. Bertoli-Barsotti, Lucio & Lando, Tommaso, 2015. "On a formula for the h-index," Journal of Informetrics, Elsevier, vol. 9(4), pages 762-776.
    18. Roth, Camille & Wu, Jiang & Lozano, Sergi, 2012. "Assessing impact and quality from local dynamics of citation networks," Journal of Informetrics, Elsevier, vol. 6(1), pages 111-120.
    19. Lafond, Francois, 2012. "Learning and the structure of citation networks," MERIT Working Papers 2012-071, United Nations University - Maastricht Economic and Social Research Institute on Innovation and Technology (MERIT).
    20. Ulrich Schmoch, 2020. "Mean values of skewed distributions in the bibliometric assessment of research units," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 925-935, November.

    More about this item

    Keywords

    Research assessment; Normalization; Scaling; Universality; Italian Universities;
    All these keywords.

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:aeg:report:2012-03. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Antonietta Angelica Zucconi (email available below). General contact details of provider: https://edirc.repec.org/data/dirosit.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.