IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v95y2013i1d10.1007_s11192-012-0854-y.html
   My bibliography  Save this article

Benchmarking research performance at the university level with information theoretic measures

Author

Listed:
  • J. A. García

    (Universidad de Granada)

  • Rosa Rodriguez-Sánchez

    (Universidad de Granada)

  • J. Fdez-Valdivia

    (Universidad de Granada)

  • Nicolas Robinson-García

    (Universidad de Granada)

  • Daniel Torres-Salinas

    (Universidad de Navarra)

Abstract

This paper presents a new method for comparing universities based on information theoretic measures. The research output of each academic institution is represented statistically by an impact-factor histogram. To this aim, for each academic institution we compute the probability of occurrence of a publication with impact factor in different intervals. Assuming the probabilities associated with a pair of academic institutions our objective is to measure the Information Gain between them. To do so, we develop an axiomatic characterization of relative information for predicting institution-institution dissimilarity. We use the Spanish university system as our scenario to test the proposed methodology for benchmarking three universities with the rest as a case study. For each case we use different scientific fields such as Information and Communication Technologies, Medicine and Pharmacy, and Economics and Business as we believe comparisons must take into account their disciplinary context. Finally we validate the Information Gain values obtained for each case with previous studies.

Suggested Citation

  • J. A. García & Rosa Rodriguez-Sánchez & J. Fdez-Valdivia & Nicolas Robinson-García & Daniel Torres-Salinas, 2013. "Benchmarking research performance at the university level with information theoretic measures," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(1), pages 435-452, April.
    • Handle: RePEc:spr:scient:v:95:y:2013:i:1:d:10.1007_s11192-012-0854-y
    DOI: 10.1007/s11192-012-0854-y
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-012-0854-y
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-012-0854-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Jerome K. Vanclay & Lutz Bornmann, 2012. "Metrics to evaluate research performance in academic institutions: a critique of ERA 2010 as applied in forestry and the indirect H2 index as a possible alternative," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(3), pages 751-771, June.
    2. Ortega, José Luis & López-Romero, Elena & Fernández, Inés, 2011. "Multivariate approach to classify research institutes according to their outputs: The case of the CSIC's institutes," Journal of Informetrics, Elsevier, vol. 5(3), pages 323-332.
    3. Henk F. Moed, 2008. "UK Research Assessment Exercises: Informed judgments on research quality or quantity?," Scientometrics, Springer;Akadémiai Kiadó, vol. 74(1), pages 153-161, January.
    4. Daniel Torres-Salinas & Jose G. Moreno-Torres & Emilio Delgado-López-Cózar & Francisco Herrera, 2011. "A methodology for Institution-Field ranking based on a bidimensional analysis: the IFQ 2 A index," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(3), pages 771-786, September.
    5. Michel Lubrano & Luc Bauwens & Alan Kirman & Camelia Protopopescu, 2003. "Ranking Economics Departments in Europe: A Statistical Approach," Journal of the European Economic Association, MIT Press, vol. 1(6), pages 1367-1401, December.
    6. Jonathan Adams & Karen Gurney & Stuart Marshall, 2007. "Profiling citation impact: A new methodology," Scientometrics, Springer;Akadémiai Kiadó, vol. 72(2), pages 325-344, August.
    7. Robert J W Tijssen & Thed N van Leeuwen & Erik van Wijk, 2009. "Benchmarking university-industry research cooperation worldwide: performance measurements and indicators based on co-authorship data for the world's largest universities," Research Evaluation, Oxford University Press, vol. 18(1), pages 13-24, March.
    8. Giovanni Abramo & Ciriaco Andrea D’Angelo & Flavia Di Costa, 2011. "National research assessment exercises: a comparison of peer review and bibliometrics rankings," Scientometrics, Springer;Akadémiai Kiadó, vol. 89(3), pages 929-941, December.
    9. J. A. García & Rosa Rodriguez-Sánchez & J. Fdez-Valdivia & Daniel Torres-Salinas & Francisco Herrera, 2012. "Ranking of research output of universities on the basis of the multidimensional prestige of influential fields: Spanish universities as a case of study," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(3), pages 1081-1099, December.
    10. J.A. García & Rosa Rodríguez-Sánchez & J. Fdez-Valdivia & N. Robinson-García & D. Torres-Salinas, 2012. "Mapping academic institutions according to their journal publication profile: Spanish universities as a case study," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(11), pages 2328-2340, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Cova, Tânia F.G.G. & Jarmelo, Susana & Formosinho, Sebastião J. & de Melo, J. Sérgio Seixas & Pais, Alberto A.C.C., 2015. "Unsupervised characterization of research institutions with task-force estimation," Journal of Informetrics, Elsevier, vol. 9(1), pages 59-68.
    2. Johannes Sorz & Wolfgang Glänzel & Ursula Ulrych & Christian Gumpenberger & Juan Gorraiz, 2020. "Research strengths identified by esteem and bibliometric indicators: a case study at the University of Vienna," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1095-1116, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Cova, Tânia F.G.G. & Jarmelo, Susana & Formosinho, Sebastião J. & de Melo, J. Sérgio Seixas & Pais, Alberto A.C.C., 2015. "Unsupervised characterization of research institutions with task-force estimation," Journal of Informetrics, Elsevier, vol. 9(1), pages 59-68.
    2. Feng Li & Yong Yi & Xiaolong Guo & Wei Qi, 2012. "Performance evaluation of research universities in Mainland China, Hong Kong and Taiwan: based on a two-dimensional approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(2), pages 531-542, February.
    3. Hiran H. Lathabai & Abhirup Nandy & Vivek Kumar Singh, 2021. "x-index: Identifying core competency and thematic research strengths of institutions using an NLP and network based ranking framework," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(12), pages 9557-9583, December.
    4. Elio Atenógenes Villaseñor & Ricardo Arencibia-Jorge & Humberto Carrillo-Calvet, 2017. "Multiparametric characterization of scientometric performance profiles assisted by neural networks: a study of Mexican higher education institutions," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(1), pages 77-104, January.
    5. Nicolás Robinson-García & Daniel Torres-Salinas & Emilio Delgado López-Cózar & Francisco Herrera, 2014. "An insight into the importance of national university rankings in an international context: the case of the I-UGR rankings of Spanish universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1309-1324, November.
    6. Nicolás Robinson-García & Clara Calero-Medina, 2014. "What do university rankings by fields rank? Exploring discrepancies between the organizational structure of universities and bibliometric classifications," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(3), pages 1955-1970, March.
    7. Gabriel-Alexandru Vîiu & Mihai Păunescu & Adrian Miroiu, 2016. "Research-driven classification and ranking in higher education: an empirical appraisal of a Romanian policy experience," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(2), pages 785-805, May.
    8. Lawson, Cornelia & Geuna, Aldo & Ana Fernández-Zubieta & Toselli, Manuel & Kataishi, Rodrigo, 2015. "International Careers of Researchers in Biomedical Sciences: A Comparison of the US and the UK," Department of Economics and Statistics Cognetti de Martiis. Working Papers 201514, University of Turin.
    9. Magnus Henrekson & Daniel Waldenström, 2011. "How Should Research Performance Be Measured? A Study Of Swedish Economists," Manchester School, University of Manchester, vol. 79(6), pages 1139-1156, December.
    10. Miriam Hein, 2005. "Wie hat sich die universitaere volkswirtschaftliche Forschung in der Schweiz seit Beginn der 90er Jahre entwickelt?�," TWI Research Paper Series 11, Thurgauer Wirtschaftsinstitut, Universität Konstanz.
    11. Justus Haucap & Michael Mödl, 2013. "Zum Verhältnis von Spitzenforschung und Politikberatung: Eine empirische Analyse vor dem Hintergrund des Ökonomenstreits," Perspektiven der Wirtschaftspolitik, Verein für Socialpolitik, vol. 14(3-4), pages 346-378, August.
    12. Miguel Sarmiento, 2010. "Central Bank Economic Research: Output, Demand, Productivity, and Relevance," Money Affairs, CEMLA, vol. 0(2), pages 211-240, July-Dece.
    13. Stephen Bazen & Patrick Moyes, 2012. "Elitism and stochastic dominance," Social Choice and Welfare, Springer;The Society for Social Choice and Welfare, vol. 39(1), pages 207-251, June.
    14. Mark J. McCabe & Christopher M. Snyder, 2015. "Does Online Availability Increase Citations? Theory and Evidence from a Panel of Economics and Business Journals," The Review of Economics and Statistics, MIT Press, vol. 97(1), pages 144-165, March.
    15. El Gibari, Samira & Gómez, Trinidad & Ruiz, Francisco, 2018. "Evaluating university performance using reference point based composite indicators," Journal of Informetrics, Elsevier, vol. 12(4), pages 1235-1250.
    16. Hans Pohl, 2021. "Internationalisation, innovation, and academic–corporate co-publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1329-1358, February.
    17. Yang, Hyeonchae & Jung, Woo-Sung, 2016. "Structural efficiency to manipulate public research institution networks," Technological Forecasting and Social Change, Elsevier, vol. 110(C), pages 21-32.
    18. Annamaria Demarinis Loiotile & Francesco De Nicolò & Adriana Agrimi & Loredana Bellantuono & Marianna La Rocca & Alfonso Monaco & Ester Pantaleo & Sabina Tangaro & Nicola Amoroso & Roberto Bellotti, 2022. "Best Practices in Knowledge Transfer: Insights from Top Universities," Sustainability, MDPI, vol. 14(22), pages 1-23, November.
    19. Hendrik P. van Dalen, 2019. "Values of Economists Matter in the Art and Science of Economics," Kyklos, Wiley Blackwell, vol. 72(3), pages 472-499, August.
    20. Angelina Keil & Peter Huber, 2004. "„Wo die Luft dünn wird…”– Zur Publikationstätigkeit der Wirtschaftsforschungsinstitute Österreichs und Deutschlands," Perspektiven der Wirtschaftspolitik, Verein für Socialpolitik, vol. 5(3), pages 363-375, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:95:y:2013:i:1:d:10.1007_s11192-012-0854-y. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.