IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v127y2022i6d10.1007_s11192-022-04377-8.html
   My bibliography  Save this article

On the possibilities of presenting the research performance of an institute over a long period of time: the case of the Institute of Information Science at the University of Graz in Austria

Author

Listed:
  • Gerhard Reichmann

    (University of Graz)

  • Christian Schlögl

    (University of Graz)

Abstract

In this paper, we demonstrate how the research performance of a university institute (department) over a long period of time can be presented and evaluated. Using the example of an information science institute at a German-speaking university, namely the (former) Institute of Information Science at the University of Graz in Austria, we present the research performance of this institute over the entire duration of its existence (33 years) in different ways. In order to be able to contextualize its performance, we compare it with that of some related institutions from all over the world. Due to the high effort involved in collecting data and the lack of data availability, the comparison must be limited to a period of a few years and—with regard to the institutions from non-German-speaking countries—to the Web of Science as data source. In this international comparison, the institute in the focus of the study shows relatively poor results. As can be seen, the choice of the data source has a major influence on the evaluation results. Especially for institutes from non-English-speaking countries with publications in their respective national languages, an exclusive use of international databases, such as Web of Science or Scopus, cannot fully consider the whole research performance. The use of personal publication lists or local research databases seems almost indispensable in these cases. A major novelty of this article is the handling of a very long evaluation period and the discussion of different ways of subdividing it. With regard to the presentation of the results, in the case of a long observation period, not only should annual and overall results be presented, but also multi-year comparisons be performed. In this way, year-by-year fluctuations can be smoothed out, and longer-term developments can be well represented.

Suggested Citation

  • Gerhard Reichmann & Christian Schlögl, 2022. "On the possibilities of presenting the research performance of an institute over a long period of time: the case of the Institute of Information Science at the University of Graz in Austria," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(6), pages 3193-3223, June.
  • Handle: RePEc:spr:scient:v:127:y:2022:i:6:d:10.1007_s11192-022-04377-8
    DOI: 10.1007/s11192-022-04377-8
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-022-04377-8
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s11192-022-04377-8?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Nabil Amara & Réjean Landry, 2012. "Counting citations in the field of business and management: why use Google Scholar rather than the Web of Science," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(3), pages 553-581, December.
    2. Alexandru T. Balaban, 2012. "Positive and negative aspects of citation indices and journal impact factors," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 241-247, August.
    3. Oliver Fabel & Miriam Hein & Robert Hofmeister, 2008. "Research Productivity in Business Economics: An Investigation of Austrian, German and Swiss Universities," German Economic Review, Verein für Socialpolitik, vol. 9(4), pages 506-531, November.
    4. Wolfgang Glänzel & Henk F. Moed, 2002. "Journal impact measures in bibliometric research," Scientometrics, Springer;Akadémiai Kiadó, vol. 53(2), pages 171-193, February.
    5. Kao, Chiang & Hung, Hsi-Tai, 2008. "Efficiency analysis of university departments: An empirical study," Omega, Elsevier, vol. 36(4), pages 653-664, August.
    6. Andrea Bonaccorsi & Tindaro Cicero, 2016. "Distributed or concentrated research excellence? Evidence from a large-scale research assessment exercise," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 67(12), pages 2976-2992, December.
    7. Mu‐hsuan Huang & Yu‐wei Chang, 2008. "Characteristics of research output in social sciences and humanities: From a research evaluation perspective," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(11), pages 1819-1828, September.
    8. Fredrik Niclas Piro & Gunnar Sivertsen, 2016. "How can differences in international university rankings be explained?," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 2263-2278, December.
    9. Beasley, J. E., 1990. "Comparing university departments," Omega, Elsevier, vol. 18(2), pages 171-183.
    10. Marlo M Vernon & E Andrew Balas & Shaher Momani, 2018. "Are university rankings useful to improve research? A systematic review," PLOS ONE, Public Library of Science, vol. 13(3), pages 1-15, March.
    11. Philippe Mongeon & Adèle Paul-Hus, 2016. "The journal coverage of Web of Science and Scopus: a comparative analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(1), pages 213-228, January.
    12. Sivertsen, Gunnar & Rousseau, Ronald & Zhang, Lin, 2019. "Measuring scientific contributions with modified fractional counting," Journal of Informetrics, Elsevier, vol. 13(2), pages 679-694.
    13. Matthias Meyer & Rüdiger W. Waldkirch & Michael A. Zaggl, 2012. "Relative Performance Measurement of Researchers: The Impact of Data Source Selection," Schmalenbach Business Review (sbr), LMU Munich School of Management, vol. 64(4), pages 308-330, October.
    14. Lorna Wildgaard & Jesper W. Schneider & Birger Larsen, 2014. "A review of the characteristics of 108 author-level bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 125-158, October.
    15. Martín-Martín, Alberto & Orduna-Malea, Enrique & Thelwall, Mike & Delgado López-Cózar, Emilio, 2018. "Google Scholar, Web of Science, and Scopus: A systematic comparison of citations in 252 subject categories," Journal of Informetrics, Elsevier, vol. 12(4), pages 1160-1177.
    16. Oliver Fabel & Miriam Hein & Robert Hofmeister, 2008. "Research Productivity in Business Economics: An Investigation of Austrian, German and Swiss Universities," German Economic Review, Verein für Socialpolitik, vol. 9(4), pages 506-531, November.
    17. Hicks, Diana, 2012. "Performance-based university research funding systems," Research Policy, Elsevier, vol. 41(2), pages 251-261.
    18. Isabelle Dorsch & Johanna M. Askeridis & Wolfgang G. Stock, 2018. "Truebounded, Overbounded, or Underbounded? Scientists’ Personal Publication Lists versus Lists Generated through Bibliographic Information Services," Publications, MDPI, vol. 6(1), pages 1-9, February.
    19. Fiona Edgar & Alan Geare, 2010. "Characteristics of High- and Low-performing University Departments as Assessed by the New Zealand Performance Based Research Funding (PBRF) Exercise," Australian Accounting Review, CPA Australia, vol. 20(1), pages 55-63, March.
    20. Lokman I. Meho & Kiduk Yang, 2007. "Impact of data sources on citation counts and rankings of LIS faculty: Web of science versus scopus and google scholar," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 58(13), pages 2105-2125, November.
    21. Berna Haktanirlar Ulutas, 2011. "Assessing the Relative Performance of University Departments: Teaching vs. Research," Istanbul University Econometrics and Statistics e-Journal, Department of Econometrics, Faculty of Economics, Istanbul University, vol. 13(1), pages 125-138, Special I.
    22. Daniel Stockemer & Michael J. Wigginton, 2019. "Publishing in English or another language: An inclusive study of scholar’s language publication preferences in the natural, social and interdisciplinary sciences," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(2), pages 645-652, February.
    23. Linda Sīle & Raf Vanderstraeten, 2019. "Measuring changes in publication patterns in a context of performance-based research funding systems: the case of educational research in the University of Gothenburg (2005–2014)," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(1), pages 71-91, January.
    24. Mike Thelwall & Nabeil Maflahi, 2020. "Academic collaboration rates and citation associations vary substantially between countries and fields," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 71(8), pages 968-978, August.
    25. Gianni De Fraja & Giovanni Facchini & John Gathergood, 2019. "Academic salaries and public evaluation of university research: Evidence from the UK Research Excellence Framework," Economic Policy, CEPR, CESifo, Sciences Po;CES;MSH, vol. 34(99), pages 523-583.
    26. Andrea Diem & Stefan C. Wolter, 2011. "The Use of Bibliometrics to Measure Research Performance in Education Sciences," Economics of Education Working Paper Series 0066, University of Zurich, Department of Business Administration (IBW), revised May 2013.
    27. Katja Rost & Bruno S. Frey, 2011. "Quantitative and Qualitative Rankings of Scholars," Schmalenbach Business Review (sbr), LMU Munich School of Management, vol. 63(1), pages 63-91, January.
    28. Juan Gorraiz & Christian Gumpenberger & Christian Schlögl, 2014. "Usage versus citation behaviours in four subject areas," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1077-1095, November.
    29. Jia Zhu & Saeed-Ul Hassan & Hamid Turab Mirza & Qing Xie, 2014. "Measuring recent research performance for Chinese universities using bibliometric methods," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 429-443, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    2. Mingers, John & Yang, Liying, 2017. "Evaluating journal quality: A review of journal citation indicators and ranking in business and management," European Journal of Operational Research, Elsevier, vol. 257(1), pages 323-337.
    3. Raminta Pranckutė, 2021. "Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World," Publications, MDPI, vol. 9(1), pages 1-59, March.
    4. Csomós, György, 2020. "Introducing recalibrated academic performance indicators in the evaluation of individuals’ research performance: A case study from Eastern Europe," Journal of Informetrics, Elsevier, vol. 14(4).
    5. Mario Fernandes & Andreas Walter, 2023. "The times they are a-changin’: profiling newly tenured business economics professors in Germany over the past thirty years," Journal of Business Economics, Springer, vol. 93(5), pages 929-971, July.
    6. Pantea Kamrani & Isabelle Dorsch & Wolfgang G. Stock, 2021. "Do researchers know what the h-index is? And how do they estimate its importance?," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 5489-5508, July.
    7. Mehdi Rhaiem & Nabil Amara, 2020. "Determinants of research efficiency in Canadian business schools: evidence from scholar-level data," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 53-99, October.
    8. Parul Khurana & Kiran Sharma, 2022. "Impact of h-index on author’s rankings: an improvement to the h-index for lower-ranked authors," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(8), pages 4483-4498, August.
    9. Martín-Martín, Alberto & Orduna-Malea, Enrique & Thelwall, Mike & Delgado López-Cózar, Emilio, 2018. "Google Scholar, Web of Science, and Scopus: A systematic comparison of citations in 252 subject categories," Journal of Informetrics, Elsevier, vol. 12(4), pages 1160-1177.
    10. Gerson Pech & Catarina Delgado, 2020. "Percentile and stochastic-based approach to the comparison of the number of citations of articles indexed in different bibliographic databases," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(1), pages 223-252, April.
    11. Maor Weinberger & Maayan Zhitomirsky-Geffet, 2021. "Diversity of success: measuring the scholarly performance diversity of tenured professors in the Israeli academia," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(4), pages 2931-2970, April.
    12. Michael Gusenbauer, 2022. "Search where you will find most: Comparing the disciplinary coverage of 56 bibliographic databases," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2683-2745, May.
    13. Adam Ayaita & Kerstin Pull & Uschi Backes-Gellner, 2019. "You get what you ‘pay’ for: academic attention, career incentives and changes in publication portfolios of business and economics researchers," Journal of Business Economics, Springer, vol. 89(3), pages 273-290, April.
    14. Yue Guiling & Siti Aisyah Panatik & Mohammad Saipol Mohd Sukor & Noraini Rusbadrol & Li Cunlin, 2022. "Bibliometric Analysis of Global Research on Organizational Citizenship Behavior From 2000 to 2019," SAGE Open, , vol. 12(1), pages 21582440221, February.
    15. Müller, Harry & Dilger, Alexander, 2013. "Der Einfluss des Forschungsschwerpunkts auf den Zitationserfolg: Eine empirische Untersuchung anhand der Gesamtpublikationen deutschsprachiger Hochschullehrer für BWL," Discussion Papers of the Institute for Organisational Economics 1/2013, University of Münster, Institute for Organisational Economics.
    16. Yaşar Tonta & Müge Akbulut, 2020. "Does monetary support increase citation impact of scholarly papers?," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1617-1641, November.
    17. Bonaccorsi, Andrea & Belingheri, Paola & Secondi, Luca, 2021. "The research productivity of universities. A multilevel and multidisciplinary analysis on European institutions," Journal of Informetrics, Elsevier, vol. 15(2).
    18. Mehdi Rhaiem, 2017. "Measurement and determinants of academic research efficiency: a systematic review of the evidence," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(2), pages 581-615, February.
    19. Groen-Xu, Moqi & Bös, Gregor & Teixeira, Pedro A. & Voigt, Thomas & Knapp, Bernhard, 2023. "Short-term incentives of research evaluations: Evidence from the UK Research Excellence Framework," Research Policy, Elsevier, vol. 52(6).
    20. Wu, Jiang & Ou, Guiyan & Liu, Xiaohui & Dong, Ke, 2022. "How does academic education background affect top researchers’ performance? Evidence from the field of artificial intelligence," Journal of Informetrics, Elsevier, vol. 16(2).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:127:y:2022:i:6:d:10.1007_s11192-022-04377-8. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.