IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v106y2016i1d10.1007_s11192-015-1788-y.html
   My bibliography  Save this article

Bibliometric analysis of output and impact based on CRIS data: a case study on the registered output of a Dutch university

Author

Listed:
  • Thed N. Leeuwen

    (Leiden University)

  • Erik Wijk

    (Leiden University)

  • Paul F. Wouters

    (Leiden University)

Abstract

In this study we combine the registered output of a whole university in the Netherlands with data retrieved from the Web of Science. The initial research question was: is it possible to show the impact of the university in its’ full broadness, taking into account the variety of disciplines covered in the research profile of the university? In order to answer this question, we analyzed the output of the university as registered in the CRIS system METIS, over the years 2004–2009. The registration covers a wide variety of scholarly outputs, and these are all taken into account in the analysis. In the study we conduct analyses on the coverage of the output of the university, both from the perspective of the output itself, towards the Web of Science (“external”), as well as from the Web of Science perspective itself (“internal”). This provides us with the necessary information to be able to draw clear conclusions on the validity of the usage of standard bibliometric methodologies in the research assessment of universities with such a research profile.

Suggested Citation

  • Thed N. Leeuwen & Erik Wijk & Paul F. Wouters, 2016. "Bibliometric analysis of output and impact based on CRIS data: a case study on the registered output of a Dutch university," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(1), pages 1-16, January.
    • Handle: RePEc:spr:scient:v:106:y:2016:i:1:d:10.1007_s11192-015-1788-y
    DOI: 10.1007/s11192-015-1788-y
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-015-1788-y
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-015-1788-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Thed N van Leeuwen, 2007. "Modelling of bibliometric approaches and importance of output verification in research performance assessment," Research Evaluation, Oxford University Press, vol. 16(2), pages 93-105, June.
    2. Moed, H. F. & Burger, W. J. M. & Frankfort, J. G. & Van Raan, A. F. J., 1985. "The use of bibliometric data for the measurement of university research performance," Research Policy, Elsevier, vol. 14(3), pages 131-149, June.
    3. Anton J. Nederhof & Thed N. Leeuwen & Anthony F. J. Raan, 2010. "Highly cited non-journal publications in political science, economics and psychology: a first exploration," Scientometrics, Springer;Akadémiai Kiadó, vol. 83(2), pages 363-374, May.
    4. Martin, Ben R. & Irvine, John, 1993. "Assessing basic research : Some partial indicators of scientific progress in radio astronomy," Research Policy, Elsevier, vol. 22(2), pages 106-106, April.
    5. Gunnar Sivertsen & Birger Larsen, 2012. "Comprehensive bibliographic coverage of the social sciences and humanities in a citation index: an empirical analysis of the potential," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(2), pages 567-575, May.
    6. Truyken L.B. Ossenblok & Frederik T. Verleysen & Tim C.E. Engels, 2014. "Coauthorship of journal articles and book chapters in the social sciences and humanities (2000–2010)," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(5), pages 882-897, May.
    7. Linda Butler & Martijn S. Visser, 2006. "Extending citation analysis to non-source items," Scientometrics, Springer;Akadémiai Kiadó, vol. 66(2), pages 327-343, February.
    8. Diana Hicks, 1999. "The difficulty of achieving full coverage of international social science literature and the bibliometric consequences," Scientometrics, Springer;Akadémiai Kiadó, vol. 44(2), pages 193-215, February.
    9. Truyken L. B. Ossenblok & Tim C. E. Engels & Gunnar Sivertsen, 2012. "The representation of the social sciences and humanities in the Web of Science--a comparison of publication patterns and incentive structures in Flanders and Norway (2005--9)," Research Evaluation, Oxford University Press, vol. 21(4), pages 280-290, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Janne Pölönen & Otto Auranen, 2022. "Research performance and scholarly communication profile of competitive research funding: the case of Academy of Finland," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 7415-7433, December.
    2. Linda Sīle & Raf Vanderstraeten, 2019. "Measuring changes in publication patterns in a context of performance-based research funding systems: the case of educational research in the University of Gothenburg (2005–2014)," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(1), pages 71-91, January.
    3. Miloš Savić & Mirjana Ivanović & Bojana Dimić Surla, 2017. "Analysis of intra-institutional research collaboration: a case of a Serbian faculty of sciences," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(1), pages 195-216, January.
    4. Colavizza, Giovanni & Franssen, Thomas & van Leeuwen, Thed, 2019. "An empirical investigation of the tribes and their territories: Are research specialisms rural and urban?," Journal of Informetrics, Elsevier, vol. 13(1), pages 105-117.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    2. Pei-Shan Chi, 2015. "Changing publication and citation patterns in political science in Germany," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 1833-1848, December.
    3. Pei-Shan Chi, 2014. "Which role do non-source items play in the social sciences? A case study in political science in Germany," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1195-1213, November.
    4. Boyack, Kevin W. & Klavans, Richard, 2014. "Including cited non-source items in a large-scale map of science: What difference does it make?," Journal of Informetrics, Elsevier, vol. 8(3), pages 569-580.
    5. Pei-Shan Chi & Stijn Conix, 2022. "Measuring the isolation of research topics in philosophy," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(4), pages 1669-1696, April.
    6. Verleysen, Frederik T. & Engels, Tim C.E., 2014. "Barycenter representation of book publishing internationalization in the Social Sciences and Humanities," Journal of Informetrics, Elsevier, vol. 8(1), pages 234-240.
    7. Mehdi Rhaiem & Nabil Amara, 2020. "Determinants of research efficiency in Canadian business schools: evidence from scholar-level data," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 53-99, October.
    8. van Raan, A. F. J. & van Leeuwen, Th. N., 2002. "Assessment of the scientific basis of interdisciplinary, applied research: Application of bibliometric methods in Nutrition and Food Research," Research Policy, Elsevier, vol. 31(4), pages 611-632, May.
    9. Lutz Bornmann & Werner Marx, 2014. "How to evaluate individual researchers working in the natural and life sciences meaningfully? A proposal of methods based on percentiles of citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(1), pages 487-509, January.
    10. Sabrina Petersohn & Thomas Heinze, 2018. "Professionalization of bibliometric research assessment. Insights from the history of the Leiden Centre for Science and Technology Studies (CWTS)," Science and Public Policy, Oxford University Press, vol. 45(4), pages 565-578.
    11. Alesia Zuccala & Roberto Cornacchia, 2016. "Data matching, integration, and interoperability for a metric assessment of monographs," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(1), pages 465-484, July.
    12. Jordi Ardanuy & Llorenç Arguimbau & Ángel Borrego, 2022. "Social sciences and humanities research funded under the European Union Sixth Framework Programme (2002–2006): a long-term assessment of projects, acknowledgements and publications," Palgrave Communications, Palgrave Macmillan, vol. 9(1), pages 1-13, December.
    13. Bertocchi, Graziella & Gambardella, Alfonso & Jappelli, Tullio & Nappi, Carmela A. & Peracchi, Franco, 2015. "Bibliometric evaluation vs. informed peer review: Evidence from Italy," Research Policy, Elsevier, vol. 44(2), pages 451-466.
    14. Bar-Ilan, Judit, 2008. "Informetrics at the beginning of the 21st century—A review," Journal of Informetrics, Elsevier, vol. 2(1), pages 1-52.
    15. Linda Sīle & Raf Vanderstraeten, 2019. "Measuring changes in publication patterns in a context of performance-based research funding systems: the case of educational research in the University of Gothenburg (2005–2014)," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(1), pages 71-91, January.
    16. Giovanni Abramo & Ciriaco Andrea D’Angelo & Myroslava Hladchenko, 2023. "Assessing the effects of publication requirements for professorship on research performance and publishing behaviour of Ukrainian academics," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(8), pages 4589-4609, August.
    17. Martin, Ben R., 2012. "The evolution of science policy and innovation studies," Research Policy, Elsevier, vol. 41(7), pages 1219-1239.
    18. Mallig, Nicolai, 2010. "A relational database for bibliometric analysis," Journal of Informetrics, Elsevier, vol. 4(4), pages 564-580.
    19. Dorte Henriksen, 2016. "The rise in co-authorship in the social sciences (1980–2013)," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(2), pages 455-476, May.
    20. Lutz Bornmann & Julian N. Marewski, 2019. "Heuristics as conceptual lens for understanding and studying the usage of bibliometrics in research evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(2), pages 419-459, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:106:y:2016:i:1:d:10.1007_s11192-015-1788-y. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.