IDEAS home Printed from https://ideas.repec.org/p/aeg/report/2014-12.html
   My bibliography  Save this paper

Beyond university rankings ? Generating new indicators on European universities by linking data in open platforms

Author

Listed:
  • Andrea Bonaccorsi

    (DESTEC, University of Pisa, Italy)

  • Cinzia Daraio

    (Department of Computer, Control and Management Engineering, Universita' degli Studi di Roma "La Sapienza")

Abstract

The need for new indicators on universities is growing as governments and decision makers at all levels are faced with the huge opportunities generated by new knowledge, but at the same time are pressed hard by budget constraints. University rankings are attracting policy and media attention, but at the same time receive harsh methodological criticism. After reviewing the critical literature on rankings, we suggest that a change in the paradigm of design and production of indicators is needed. The traditional approach is one that leverages on existing data but also suggests heavy investment to integrate existing databases and build up tailored indicators. We show how the intelligent integration of existing data may lead to an open linked data platform that permits the construction of new indicators. The power of the approach derives from the ability to combine heterogeneous sources of data in order to derive indicators that address a variety of user needs,without designing the indicators on a custom basis.

Suggested Citation

  • Andrea Bonaccorsi & Cinzia Daraio, 2014. "Beyond university rankings ? Generating new indicators on European universities by linking data in open platforms," DIAG Technical Reports 2014-12, Department of Computer, Control and Management Engineering, Universita' degli Studi di Roma "La Sapienza".
    • Handle: RePEc:aeg:report:2014-12
    as

    Download full text from publisher

    File URL: http://www.dis.uniroma1.it/~bibdis/RePEc/aeg/report/2014-12.pdf
    File Function: First version, 2014
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Bonaccorsi, Andrea & Daraio, Cinzia & Simar, Leopold, 2014. "Efficiency and economies of scale and scope in European universities: a directional distance approach," LIDAM Discussion Papers ISBA 2014020, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    2. Stewart, Jenny, 1995. "Models of priority-setting for public sector research," Research Policy, Elsevier, vol. 24(1), pages 115-126, January.
    3. Daraio, Cinzia & Bonaccorsi, Andrea & Simar, Léopold, 2015. "Rankings and university performance: A conditional multidimensional approach," European Journal of Operational Research, Elsevier, vol. 244(3), pages 918-930.
    4. Isabelle Huault & V. Perret & S. Charreire-Petit, 2007. "Management," Post-Print halshs-00337676, HAL.
    5. Grewal, Rajdeep & Dearden, James A. & Lilien, Gary L., 2008. "The University Rankings Game: Modeling the Competition Among Universities for Ranking," The American Statistician, American Statistical Association, vol. 62, pages 232-237, August.
    6. Blaise Cronin, 2013. "Thinking about data," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(3), pages 435-436, March.
    7. Thed N. Van Leeuwen & Martijn S. Visser & Henk F. Moed & Ton J. Nederhof & Anthony F. J. Van Raan, 2003. "The Holy Grail of science policy: Exploring and combining bibliometric tools in search of scientific excellence," Scientometrics, Springer;Akadémiai Kiadó, vol. 57(2), pages 257-280, June.
    8. Ellen Hazelkorn, 2007. "The Impact of League Tables and Ranking Systems on Higher Education Decision Making," Higher Education Management and Policy, OECD Publishing, vol. 19(2), pages 1-24.
    9. Daraio, Cinzia & Bonaccorsi, Andrea & Geuna, Aldo & Lepori, Benedetto & Bach, Laurent & Bogetoft, Peter & F. Cardoso, Margarida & Castro-Martinez, Elena & Crespi, Gustavo & de Lucio, Ignacio Fernandez, 2011. "The European university landscape: A micro characterization based on evidence from the Aquameth project," Research Policy, Elsevier, vol. 40(1), pages 148-164, February.
    10. Jamil Salmi & Alenoush Saroyan, 2007. "League Tables as Policy Instruments: Uses and Misuses," Higher Education Management and Policy, OECD Publishing, vol. 19(2), pages 1-38.
    11. Andrea Bonaccorsi & Cinzia Daraio (ed.), 2007. "Universities and Strategic Knowledge Creation," Books, Edward Elgar Publishing, number 12626.
    12. Robert J. W. Tijssen & Martijn S. Visser & Thed N. van Leeuwen, 2002. "Benchmarking international scientific excellence: Are highly cited research papers an appropriate frame of reference?," Scientometrics, Springer;Akadémiai Kiadó, vol. 54(3), pages 381-397, July.
    13. Lutz Bornmann, 2014. "On the function of university rankings," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(2), pages 428-429, February.
    14. Ellen Hazelkorn, 2009. "Rankings and the battle for world-class excellence: Institutional strategies and policy choices," Higher Education Management and Policy, OECD Publishing, vol. 21(1), pages 1-22.
    15. Andrea Bonaccorsi & Cinzia Daraio, 2008. "The differentiation of the strategic profile of higher education institutions. New positioning indicators based on microdata," Scientometrics, Springer;Akadémiai Kiadó, vol. 74(1), pages 15-37, January.
    16. Saisana, Michaela & d'Hombres, Béatrice & Saltelli, Andrea, 2011. "Rickety numbers: Volatility of university rankings and policy implications," Research Policy, Elsevier, vol. 40(1), pages 165-177, February.
    17. Isidro F. Aguillo & Judit Bar-Ilan & Mark Levene & José Luis Ortega, 2010. "Comparing university rankings," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(1), pages 243-256, October.
    18. Jamil Salmi, 2009. "The Challenge of Establishing World-Class Universities," World Bank Publications - Books, The World Bank Group, number 2600, December.
    19. Vicente Safón, 2013. "What do global university rankings really measure? The search for the X factor and the X entity," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(2), pages 223-244, November.
    20. Kuang-hua Chen & Pei-yu Liao, 2012. "A comparative study on world university rankings: a bibliometric survey," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(1), pages 89-103, July.
    21. Blaise Cronin, 2013. "Thinking about data," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(3), pages 435-436, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Daraio, Cinzia & Bonaccorsi, Andrea & Simar, Léopold, 2015. "Rankings and university performance: A conditional multidimensional approach," European Journal of Operational Research, Elsevier, vol. 244(3), pages 918-930.
    2. Bonaccorsi, Andrea & Cicero, Tindaro, 2016. "Nondeterministic ranking of university departments," Journal of Informetrics, Elsevier, vol. 10(1), pages 224-237.
    3. Ibrahim Shehatta & Khalid Mahmood, 2016. "Correlation among top 100 universities in the major six global rankings: policy implications," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1231-1254, November.
    4. Ludo Waltman & Clara Calero-Medina & Joost Kosten & Ed C.M. Noyons & Robert J.W. Tijssen & Nees Jan Eck & Thed N. Leeuwen & Anthony F.J. Raan & Martijn S. Visser & Paul Wouters, 2012. "The Leiden ranking 2011/2012: Data collection, indicators, and interpretation," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(12), pages 2419-2432, December.
    5. Cinzia Daraio, 2017. "A framework for the Assessment of Research and its impacts," DIAG Technical Reports 2017-04, Department of Computer, Control and Management Engineering, Universita' degli Studi di Roma "La Sapienza".
    6. Mehdi Rhaiem & Nabil Amara, 2020. "Determinants of research efficiency in Canadian business schools: evidence from scholar-level data," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 53-99, October.
    7. Murat Perit Çakır & Cengiz Acartürk & Oğuzhan Alaşehir & Canan Çilingir, 2015. "A comparative analysis of global and national university ranking systems," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(3), pages 813-848, June.
    8. Christopher Claassen, 2015. "Measuring university quality," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(3), pages 793-807, September.
    9. Olcay, Gokcen Arkali & Bulu, Melih, 2017. "Is measuring the knowledge creation of universities possible?: A review of university rankings," Technological Forecasting and Social Change, Elsevier, vol. 123(C), pages 153-160.
    10. A. Ferrer-Sapena & E. Erdogan & E Jiménez-Fernández & E. A. Sánchez-Pérez & F. Peset, 2020. "Self-defined information indices: application to the case of university rankings," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(3), pages 2443-2456, September.
    11. Cinzia Daraio & Maurizio Lenzerini & Claudio Leporelli & Henk F. Moed & Paolo Naggar & Andrea Bonaccorsi & Alessandro Bartolucci, 2015. "Data Integration for Research and Innovation Policy: An Ontology-based Data Management Approach," DIAG Technical Reports 2015-10, Department of Computer, Control and Management Engineering, Universita' degli Studi di Roma "La Sapienza".
    12. Vicente Safón, 2013. "What do global university rankings really measure? The search for the X factor and the X entity," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(2), pages 223-244, November.
    13. Jacek Pietrucha, 2018. "Country-specific determinants of world university rankings," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 1129-1139, March.
    14. Cinzia Daraio & Maurizio Lenzerini & Claudio Leporelli & Henk F. Moed & Paolo Naggar & Andrea Bonaccorsi & Alessandro Bartolucci, 2016. "Data integration for research and innovation policy: an Ontology-Based Data Management approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(2), pages 857-871, February.
    15. Antonio Fernández-Cano & Elvira Curiel-Marin & Manuel Torralbo-Rodríguez & Mónica Vallejo-Ruiz, 2018. "Questioning the Shanghai Ranking methodology as a tool for the evaluation of universities: an integrative review," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 2069-2083, September.
    16. F. Cugnata & G. Perucca & S. Salini, 2017. "Bayesian networks and the assessment of universities' value added," Journal of Applied Statistics, Taylor & Francis Journals, vol. 44(10), pages 1785-1806, July.
    17. Bjørn Stensaker, 2018. "University Alliances: Enhancing Control, Capacity, and Creativity in Dynamic Environments," Voprosy obrazovaniya / Educational Studies Moscow, National Research University Higher School of Economics, issue 1, pages 132-153.
    18. Rodríguez-Navarro, Alonso & Brito, Ricardo, 2018. "Technological research in the EU is less efficient than the world average. EU research policy risks Europeans’ future," Journal of Informetrics, Elsevier, vol. 12(3), pages 718-731.
    19. Bonaccorsi, Andrea & Haddawy, Peter & Cicero, Tindaro & Hassan, Saeed-Ul, 2017. "The solitude of stars. An analysis of the distributed excellence model of European universities," Journal of Informetrics, Elsevier, vol. 11(2), pages 435-454.
    20. Ludo Waltman & Michael Schreiber, 2013. "On the calculation of percentile-based bibliometric indicators," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(2), pages 372-379, February.

    More about this item

    Keywords

    university; rankings; indicators design; data integration;
    All these keywords.

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:aeg:report:2014-12. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Antonietta Angelica Zucconi (email available below). General contact details of provider: https://edirc.repec.org/data/dirosit.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.