IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v107y2016i3d10.1007_s11192-016-1935-0.html
   My bibliography  Save this article

Designing a Composite Index for research performance evaluation at the national or regional level: ranking Central Universities in India

Author

Listed:
  • Aparna Basu

    (CSIR-NISTADS)

  • Sumit Kumar Banshal

    (South Asian University)

  • Khushboo Singhal

    (South Asian University)

  • Vivek Kumar Singh

    (Banaras Hindu University)

Abstract

It is now generally accepted that institutions of higher education and research, largely publicly funded, need to be subjected to some benchmarking process or performance evaluation. Currently there are several international ranking exercises that rank institutions at the global level, using a variety of performance criteria such as research publication data, citations, awards and reputation surveys etc. In these ranking exercises, the data are combined in specified ways to create an index which is then used to rank the institutions. These lists are generally limited to the top 500–1000 institutions in the world. Further, some criteria (e.g., the Nobel Prize), used in some of the ranking exercises, are not relevant for the large number of institutions that are in the medium range. In this paper we propose a multidimensional ‘Quality–Quantity’ Composite Index for a group of institutions using bibliometric data, that can be used for ranking and for decision making or policy purposes at the national or regional level. The index is applied here to rank Central Universities in India. The ranks obtained compare well with those obtained with the h-index and partially with the size-dependent Leiden ranking and University Ranking by Academic Performance. A generalized model for the index using other variables and variable weights is proposed.

Suggested Citation

  • Aparna Basu & Sumit Kumar Banshal & Khushboo Singhal & Vivek Kumar Singh, 2016. "Designing a Composite Index for research performance evaluation at the national or regional level: ranking Central Universities in India," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(3), pages 1171-1193, June.
  • Handle: RePEc:spr:scient:v:107:y:2016:i:3:d:10.1007_s11192-016-1935-0
    DOI: 10.1007/s11192-016-1935-0
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-016-1935-0
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s11192-016-1935-0?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Aparna Basu & Ritu Aggarwal, 2001. "International Collaboration in Science in India and its Impact on Institutional Performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 52(3), pages 379-394, November.
    2. Veljko Jeremic & Milica Bulajic & Milan Martic & Zoran Radojicic, 2011. "A fresh approach to evaluating the academic ranking of world universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(3), pages 587-596, June.
    3. Ludo Waltman & Nees Jan Eck, 2012. "A new methodology for constructing a publication-level classification system of science," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(12), pages 2378-2392, December.
    4. P. Nishy & Yatish Panwar & Suresh Prasad & G. K. Mandal & Gangan Prathap, 2012. "An impact-citations-exergy (iCX) trajectory analysis of leading research institutions in India," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(1), pages 245-251, April.
    5. Jean-Charles Billaut & Denis Bouyssou & Philippe Vincke, 2010. "Should you believe in the Shanghai ranking?," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(1), pages 237-263, July.
    6. Themis Lazaridis, 2010. "Ranking university departments using the mean h-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(2), pages 211-216, February.
    7. Jean-Charles Billaut & Denis Bouyssou & Philippe Vincke, 2010. "Should you believe in the Shanghai ranking?," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(1), pages 237-263, July.
    8. Ashraf Uddin & Jaideep Bhoosreddy & Marisha Tiwari & Vivek Kumar Singh, 2016. "A Sciento-text framework to characterize research strength of institutions at fine-grained thematic area level," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(3), pages 1135-1150, March.
    9. Marco Geraci & M. Degli Esposti, 2011. "Where do Italian universities stand? An in-depth statistical analysis of national and international rankings," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(3), pages 667-681, June.
    10. Lutz Bornmann, 2013. "How to analyze percentile citation impact data meaningfully in bibliometrics: The statistical analysis of distributions, percentile rank classes, and top-cited papers," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(3), pages 587-595, March.
    11. repec:dau:papers:123456789/2947 is not listed on IDEAS
    12. Vivek Kumar Singh & Ashraf Uddin & David Pinto, 2015. "Computer science research: the top 100 institutions in India and in the world," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(2), pages 529-553, August.
    13. Ludo Waltman & Clara Calero-Medina & Joost Kosten & Ed C.M. Noyons & Robert J.W. Tijssen & Nees Jan Eck & Thed N. Leeuwen & Anthony F.J. Raan & Martijn S. Visser & Paul Wouters, 2012. "The Leiden ranking 2011/2012: Data collection, indicators, and interpretation," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(12), pages 2419-2432, December.
    14. Ludo Waltman & Nees Jan van Eck, 2012. "A new methodology for constructing a publication‐level classification system of science," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(12), pages 2378-2392, December.
    15. Lutz Bornmann & Werner Marx, 2014. "How to evaluate individual researchers working in the natural and life sciences meaningfully? A proposal of methods based on percentiles of citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(1), pages 487-509, January.
    16. Anthony F. J. van Raan, 2005. "Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods," Scientometrics, Springer;Akadémiai Kiadó, vol. 62(1), pages 133-143, January.
    17. Péter Vinkler, 2006. "Composite scientometric indicators for evaluating publications of research institutes," Scientometrics, Springer;Akadémiai Kiadó, vol. 68(3), pages 629-642, September.
    18. Bornmann, Lutz & Stefaner, Moritz & de Moya Anegón, Felix & Mutz, Rüdiger, 2014. "What is the effect of country-specific characteristics on the research performance of scientific institutions? Using multi-level statistical models to rank and map universities and research-focused in," Journal of Informetrics, Elsevier, vol. 8(3), pages 581-593.
    19. Nian Cai Liu & Ying Cheng & Li Liu, 2005. "Academic ranking of world universities using scientometrics - A comment to the “Fatal Attraction”," Scientometrics, Springer;Akadémiai Kiadó, vol. 64(1), pages 101-109, July.
    20. Alain Molinari & Jean-Francois Molinari, 2008. "Mathematical aspects of a new criterion for ranking scientific institutions based on the h-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 75(2), pages 339-356, May.
    21. Alan Peter Matthews, 2012. "South African universities in world rankings," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(3), pages 675-695, September.
    22. Costas, Rodrigo & Bordons, María, 2007. "The h-index: Advantages, limitations and its relation with other bibliometric indicators at the micro level," Journal of Informetrics, Elsevier, vol. 1(3), pages 193-203.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hiran H. Lathabai & Abhirup Nandy & Vivek Kumar Singh, 2021. "x-index: Identifying core competency and thematic research strengths of institutions using an NLP and network based ranking framework," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(12), pages 9557-9583, December.
    2. Ashraf Uddin & Jaideep Bhoosreddy & Marisha Tiwari & Vivek Kumar Singh, 2016. "A Sciento-text framework to characterize research strength of institutions at fine-grained thematic area level," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(3), pages 1135-1150, March.
    3. Franceschini, Fiorenzo & Maisano, Domenico, 2011. "Structured evaluation of the scientific output of academic research groups by recent h-based indicators," Journal of Informetrics, Elsevier, vol. 5(1), pages 64-74.
    4. Shahryar Rahnamayan & Sedigheh Mahdavi & Kalyanmoy Deb & Azam Asilian Bidgoli, 2020. "Ranking Multi-Metric Scientific Achievements Using a Concept of Pareto Optimality," Mathematics, MDPI, vol. 8(6), pages 1-46, June.
    5. Massucci, Francesco Alessandro & Docampo, Domingo, 2019. "Measuring the academic reputation through citation networks via PageRank," Journal of Informetrics, Elsevier, vol. 13(1), pages 185-201.
    6. Leo Freyer, 2014. "Robust rankings," Scientometrics, Springer;Akadémiai Kiadó, vol. 100(2), pages 391-406, August.
    7. Milica Jovanovic & Veljko Jeremic & Gordana Savic & Milica Bulajic & Milan Martic, 2012. "How does the normalization of data affect the ARWU ranking?," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(2), pages 319-327, November.
    8. Bouyssou, Denis & Marchant, Thierry, 2016. "Ranking authors using fractional counting of citations: An axiomatic approach," Journal of Informetrics, Elsevier, vol. 10(1), pages 183-199.
    9. Antonio Fernández-Cano & Elvira Curiel-Marin & Manuel Torralbo-Rodríguez & Mónica Vallejo-Ruiz, 2018. "Questioning the Shanghai Ranking methodology as a tool for the evaluation of universities: an integrative review," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 2069-2083, September.
    10. D. Docampo & D. Egret & L. Cram, 2015. "The effect of university mergers on the Shanghai ranking," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(1), pages 175-191, July.
    11. Gul, Muhammet & Yucesan, Melih, 2022. "Performance evaluation of Turkish Universities by an integrated Bayesian BWM-TOPSIS model," Socio-Economic Planning Sciences, Elsevier, vol. 80(C).
    12. Abramo, Giovanni & D’Angelo, Ciriaco Andrea, 2015. "Evaluating university research: Same performance indicator, different rankings," Journal of Informetrics, Elsevier, vol. 9(3), pages 514-525.
    13. Enar Ruiz-Conde & Aurora Calderón-Martínez, 2014. "University institutional repositories: competitive environment and their role as communication media of scientific knowledge," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(2), pages 1283-1299, February.
    14. Veljko Jeremic & Milica Bulajic & Milan Martic & Zoran Radojicic, 2011. "A fresh approach to evaluating the academic ranking of world universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(3), pages 587-596, June.
    15. Mingers, John & Leydesdorff, Loet, 2015. "A review of theory and practice in scientometrics," European Journal of Operational Research, Elsevier, vol. 246(1), pages 1-19.
    16. Esteban Fernández Tuesta & Máxima Bolaños-Pizarro & Daniel Pimentel Neves & Geziel Fernández & Justin Axel-Berg, 2020. "Complex networks for benchmarking in global universities rankings," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 405-425, October.
    17. Juntao Zheng & Niancai Liu, 2015. "Mapping of important international academic awards," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(3), pages 763-791, September.
    18. Tahamtan, Iman & Bornmann, Lutz, 2018. "Core elements in the process of citing publications: Conceptual overview of the literature," Journal of Informetrics, Elsevier, vol. 12(1), pages 203-216.
    19. Bornmann, Lutz & Haunschild, Robin & Mutz, Rüdiger, 2020. "Should citations be field-normalized in evaluative bibliometrics? An empirical analysis based on propensity score matching," Journal of Informetrics, Elsevier, vol. 14(4).
    20. Ludo Waltman & Clara Calero-Medina & Joost Kosten & Ed C.M. Noyons & Robert J.W. Tijssen & Nees Jan Eck & Thed N. Leeuwen & Anthony F.J. Raan & Martijn S. Visser & Paul Wouters, 2012. "The Leiden ranking 2011/2012: Data collection, indicators, and interpretation," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(12), pages 2419-2432, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:107:y:2016:i:3:d:10.1007_s11192-016-1935-0. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.