IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v119y2019i2d10.1007_s11192-019-03089-w.html
   My bibliography  Save this article

A new approach to the analysis and evaluation of the research output of countries and institutions

Author

Listed:
  • Domingo Docampo

    (University of Vigo)

  • Jean-Jacques Bessoule

    (UMR 5200, CNRS – Univ. Bordeaux)

Abstract

A plethora of bibliometric indicators is available nowadays to gauge research performance. The spectrum of bibliometric based measures is very broad, from purely size-dependent indicators (e.g. raw counts of scientific contributions and/or citations) up to size-independent measures (e.g. citations per paper, publications or citations per researcher), through a number of indicators that effectively combine quantitative and qualitative features (e.g. the h-index). In this paper we present a straightforward procedure to evaluate the scientific contribution of territories and institutions that combines size-dependent and scale-free measures. We have analysed in the paper the scientific production of 189 countries in the period 2006–2015. Our approach enables effective global and field-related comparative analyses of the scientific productions of countries and academic/research institutions. Furthermore, the procedure helps to identifying strengths and weaknesses of a given country or institution, by tracking variations of performance ratios across research fields. Moreover, by using a straightforward wealth-index, we show how research performance measures are highly associated with the wealth of countries and territories. Given the simplicity of the methods introduced in this paper and the fact that their results are easily understandable by non-specialists, we believe they could become a useful tool for the assessment of the research output of countries and institutions.

Suggested Citation

  • Domingo Docampo & Jean-Jacques Bessoule, 2019. "A new approach to the analysis and evaluation of the research output of countries and institutions," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(2), pages 1207-1225, May.
    • Handle: RePEc:spr:scient:v:119:y:2019:i:2:d:10.1007_s11192-019-03089-w
    DOI: 10.1007/s11192-019-03089-w
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-019-03089-w
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-019-03089-w?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Vesna Luzar & Vesna Dobrić & Siniša Maričić & Greta Pifat & Jagoda Spaventi, 1992. "A methodology for cluster analysis of citation histories," Quality & Quantity: International Journal of Methodology, Springer, vol. 26(4), pages 337-365, November.
    2. Wolfgang Glänzel & Henk F. Moed, 2002. "Journal impact measures in bibliometric research," Scientometrics, Springer;Akadémiai Kiadó, vol. 53(2), pages 171-193, February.
    3. Gangan Prathap, 2011. "Quasity, when quantity has a quality all of its own—toward a theory of performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(2), pages 555-562, August.
    4. Gangan Prathap, 2010. "The 100 most prolific economists using the p-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(1), pages 167-172, July.
    5. Gangan Prathap, 2010. "Is there a place for a mock h-index?," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(1), pages 153-165, July.
    6. Lorna Wildgaard & Jesper W. Schneider & Birger Larsen, 2014. "A review of the characteristics of 108 author-level bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 125-158, October.
    7. Zhenbin Yan & Qiang Wu & Xingchen Li, 2016. "Do Hirsch-type indices behave the same in assessing single publications? An empirical study of 29 bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1815-1833, December.
    8. Abramo, Giovanni & D’Angelo, Ciriaco Andrea, 2016. "A farewell to the MNCS and like size-independent indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 646-651.
    9. Leo Egghe, 2006. "Theory and practise of the g-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 69(1), pages 131-152, October.
    10. Christoph Emanuel Mueller, 2016. "Accurate forecast of countries’ research output by macro-level indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1307-1328, November.
    11. Ammar Nejati & Seyyed Mehdi Hosseini Jenab, 2010. "A two-dimensional approach to evaluate the scientific production of countries (case study: the basic sciences)," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(2), pages 357-364, August.
    12. Loet Leydesdorff, 2009. "How are new citation‐based journal indicators adding to the bibliometric toolbox?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(7), pages 1327-1336, July.
    13. Schreiber, M. & Malesios, C.C. & Psarakis, S., 2012. "Exploratory factor analysis for the Hirsch index, 17 h-type variants, and some traditional bibliometric indicators," Journal of Informetrics, Elsevier, vol. 6(3), pages 347-358.
    14. Aksnes, Dag W. & Schneider, Jesper W. & Gunnarsson, Magnus, 2012. "Ranking national research systems by citation indicators. A comparative analysis using whole and fractionalised counting methods," Journal of Informetrics, Elsevier, vol. 6(1), pages 36-43.
    15. Cabrerizo, F.J. & Alonso, S. & Herrera-Viedma, E. & Herrera, F., 2010. "q2-Index: Quantitative and qualitative evaluation based on the number and impact of papers in the Hirsch core," Journal of Informetrics, Elsevier, vol. 4(1), pages 23-28.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mehmet Pinar, 2023. "Do research performances of universities and disciplines in England converge or diverge? An assessment of the progress between research excellence frameworks in 2014 and 2021," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(10), pages 5731-5766, October.
    2. Tanel Hirv, 2022. "The interplay of the size of the research system, ways of collaboration, level, and method of funding in determining bibliometric outputs," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(3), pages 1295-1316, March.
    3. Wolfgang G. Stock & Isabelle Dorsch & Gerhard Reichmann & Christian Schlögl, 2023. "Labor productivity, labor impact, and co-authorship of research institutions: publications and citations per full-time equivalents," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(1), pages 363-377, January.
    4. Alonso Rodríguez-Navarro & Ricardo Brito, 2022. "The link between countries’ economic and scientific wealth has a complex dependence on technological activity and research policy," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2871-2896, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    2. Sidiropoulos, A. & Gogoglou, A. & Katsaros, D. & Manolopoulos, Y., 2016. "Gazing at the skyline for star scientists," Journal of Informetrics, Elsevier, vol. 10(3), pages 789-813.
    3. Deming Lin & Tianhui Gong & Wenbin Liu & Martin Meyer, 2020. "An entropy-based measure for the evolution of h index research," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2283-2298, December.
    4. Wei, Shelia X. & Tong, Tong & Rousseau, Ronald & Wang, Wanru & Ye, Fred Y., 2022. "Relations among the h-, g-, ψ-, and p-index and offset-ability," Journal of Informetrics, Elsevier, vol. 16(4).
    5. Vîiu, Gabriel-Alexandru, 2016. "A theoretical evaluation of Hirsch-type bibliometric indicators confronted with extreme self-citation," Journal of Informetrics, Elsevier, vol. 10(2), pages 552-566.
    6. Georgios Stoupas & Antonis Sidiropoulos & Antonia Gogoglou & Dimitrios Katsaros & Yannis Manolopoulos, 2018. "Rainbow ranking: an adaptable, multidimensional ranking method for publication sets," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(1), pages 147-160, July.
    7. Franceschini, Fiorenzo & Maisano, Domenico, 2010. "The Hirsch spectrum: A novel tool for analyzing scientific journals," Journal of Informetrics, Elsevier, vol. 4(1), pages 64-73.
    8. Gangan Prathap, 2018. "Eugene Garfield: from the metrics of science to the science of metrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(2), pages 637-650, February.
    9. Gangan Prathap, 2014. "Single parameter indices and bibliometric outliers," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(3), pages 1781-1787, December.
    10. Péter Vinkler, 2019. "Core journals and elite subsets in scientometrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(1), pages 241-259, October.
    11. Muhammad Raheel & Samreen Ayaz & Muhammad Tanvir Afzal, 2018. "Evaluation of h-index, its variants and extensions based on publication age & citation intensity in civil engineering," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 1107-1127, March.
    12. Abdulrahman A. Alshdadi & Muhammad Usman & Madini O. Alassafi & Muhammad Tanvir Afzal & Rayed AlGhamdi, 2023. "Formulation of rules for the scientific community using deep learning," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(3), pages 1825-1852, March.
    13. Lucio Bertoli-Barsotti & Tommaso Lando, 2017. "The h-index as an almost-exact function of some basic statistics," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 1209-1228, November.
    14. Vîiu, Gabriel-Alexandru, 2017. "Disaggregated research evaluation through median-based characteristic scores and scales: a comparison with the mean-based approach," Journal of Informetrics, Elsevier, vol. 11(3), pages 748-765.
    15. Lorna Wildgaard, 2015. "A comparison of 17 author-level bibliometric indicators for researchers in Astronomy, Environmental Science, Philosophy and Public Health in Web of Science and Google Scholar," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(3), pages 873-906, September.
    16. Marek Gagolewski & Barbara Żogała-Siudem & Grzegorz Siudem & Anna Cena, 2022. "Fairness in the three-dimensional model for citation impact," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(10), pages 6055-6059, October.
    17. R. Karpagam & S. Gopalakrishnan & M. Natarajan & B. Ramesh Babu, 2011. "Mapping of nanoscience and nanotechnology research in India: a scientometric analysis, 1990–2009," Scientometrics, Springer;Akadémiai Kiadó, vol. 89(2), pages 501-522, November.
    18. Zhenbin Yan & Qiang Wu & Xingchen Li, 2016. "Do Hirsch-type indices behave the same in assessing single publications? An empirical study of 29 bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1815-1833, December.
    19. Thor, Andreas & Marx, Werner & Leydesdorff, Loet & Bornmann, Lutz, 2016. "Introducing CitedReferencesExplorer (CRExplorer): A program for reference publication year spectroscopy with cited references standardization," Journal of Informetrics, Elsevier, vol. 10(2), pages 503-515.
    20. Christoph Bartneck & Servaas Kokkelmans, 2011. "Detecting h-index manipulation through self-citation analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(1), pages 85-98, April.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:119:y:2019:i:2:d:10.1007_s11192-019-03089-w. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.