IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v8y2014i3p581-593.html
   My bibliography  Save this article

What is the effect of country-specific characteristics on the research performance of scientific institutions? Using multi-level statistical models to rank and map universities and research-focused institutions worldwide

Author

Listed:
  • Bornmann, Lutz
  • Stefaner, Moritz
  • de Moya Anegón, Felix
  • Mutz, Rüdiger

Abstract

Bornmann, Stefaner, de Moya Anegón, and Mutz (2014) have introduced a web application (www.excellencemapping.net) which is linked to both academic ranking lists published hitherto (e.g. the Academic Ranking of World Universities) as well as spatial visualization approaches. The web application visualizes institutional performance within specific subject areas as ranking lists and on custom tile-based maps. The new, substantially enhanced version of the web application and the generalized linear mixed model for binomial data on which it is based are described in this paper. Scopus data are used which have been collected for the SCImago Institutions Ranking. Only those universities and research-focused institutions are considered that have published at least 500 articles, reviews and conference papers in the period 2006–2010 in a certain Scopus subject area. In the enhanced version, the effect of single covariates (such as the per capita GDP of a country in which an institution is located) on two performance metrics (best paper rate and best journal rate) is examined and visualized. A covariate-adjusted ranking and mapping of the institutions is produced in which the single covariates are held constant. The results on the performance of institutions can then be interpreted as if the institutions all had the same value (reference point) for the covariate in question. For example, those institutions can be identified worldwide showing a very good performance despite a bad financial situation in the corresponding country.

Suggested Citation

  • Bornmann, Lutz & Stefaner, Moritz & de Moya Anegón, Felix & Mutz, Rüdiger, 2014. "What is the effect of country-specific characteristics on the research performance of scientific institutions? Using multi-level statistical models to rank and map universities and research-focused in," Journal of Informetrics, Elsevier, vol. 8(3), pages 581-593.
    • Handle: RePEc:eee:infome:v:8:y:2014:i:3:p:581-593
    DOI: 10.1016/j.joi.2014.04.008
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157714000479
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.joi.2014.04.008?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Bornmann, Lutz & Leydesdorff, Loet & Wang, Jian, 2014. "How to improve the prediction based on citation impact percentiles for years shortly after the publication date?," Journal of Informetrics, Elsevier, vol. 8(1), pages 175-180.
    2. Bornmann, Lutz & Leydesdorff, Loet & Wang, Jian, 2013. "Which percentile-based approach should be preferred for calculating normalized citation impact values? An empirical comparison of five approaches including a newly developed citation-rank approach (P1," Journal of Informetrics, Elsevier, vol. 7(4), pages 933-944.
    3. Bornmann, Lutz & Leydesdorff, Loet & Walch-Solimena, Christiane & Ettl, Christoph, 2011. "Mapping excellence in the geography of science: An approach based on Scopus data," Journal of Informetrics, Elsevier, vol. 5(4), pages 537-546.
    4. Rolf Ketzler & Klaus F. Zimmermann, 2013. "A citation-analysis of economic research institutes," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(3), pages 1095-1112, June.
    5. Bornmann, Lutz & Leydesdorff, Loet & Mutz, Rüdiger, 2013. "The use of percentiles and percentile rank classes in the analysis of bibliometric data: Opportunities and limits," Journal of Informetrics, Elsevier, vol. 7(1), pages 158-165.
    6. Richard Van Noorden, 2010. "Cities: Building the best cities for science," Nature, Nature, vol. 467(7318), pages 906-908, October.
    7. Guerrero-Bote, Vicente P. & Moya-Anegón, Félix, 2012. "A further step forward in measuring journals’ scientific prestige: The SJR2 indicator," Journal of Informetrics, Elsevier, vol. 6(4), pages 674-688.
    8. Bornmann, Lutz & Waltman, Ludo, 2011. "The detection of “hot regions” in the geography of science—A visualization approach by using density maps," Journal of Informetrics, Elsevier, vol. 5(4), pages 547-553.
    9. Lutz Bornmann & Rüdiger Mutz & Hans‐Dieter Daniel, 2013. "Multilevel‐statistical reformulation of citation‐based university rankings: The Leiden ranking 2011/2012," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(8), pages 1649-1658, August.
    10. Bárbara S. Lancho-Barrantes & Vicente P. Guerrero-Bote & Félix Moya-Anegón, 2013. "Citation increments between collaborating countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(3), pages 817-831, March.
    11. Daniel Bauer, 2009. "A Note on Comparing the Estimates of Models for Cluster-Correlated or Longitudinal Data with Binary or Ordinal Outcomes," Psychometrika, Springer;The Psychometric Society, vol. 74(1), pages 97-105, March.
    12. Lutz Bornmann & Loet Leydesdorff, 2011. "Which cities produce more excellent papers than can be expected? A new mapping approach, using Google Maps, based on statistical significance testing," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(10), pages 1954-1962, October.
    13. Harzing, Anne-Wil & Giroud, Axèle, 2014. "The competitive advantage of nations: An application to academia," Journal of Informetrics, Elsevier, vol. 8(1), pages 29-42.
    14. Jonathan Adams, 2013. "The fourth age of research," Nature, Nature, vol. 497(7451), pages 557-560, May.
    15. Giovanni Abramo & Tindaro Cicero & Ciriaco Andrea D’Angelo, 2013. "National peer-review research assessment exercises for the hard sciences can be a complete waste of money: the Italian case," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(1), pages 311-324, April.
    16. Sultan Ayoub Meo & Abeer A Al Masri & Adnan Mahmood Usmani & Almas Naeem Memon & Syed Ziauddin Zaidi, 2013. "Impact of GDP, Spending on R&D, Number of Universities and Scientific Journals on Research Publications among Asian Countries," PLOS ONE, Public Library of Science, vol. 8(6), pages 1-8, June.
    17. Lutz Bornmann & Rüdiger Mutz & Werner Marx & Hermann Schier & Hans‐Dieter Daniel, 2011. "A multilevel modelling approach to investigating the predictive validity of editorial decisions: do the editors of a high profile journal select manuscripts that are highly cited after publication?," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 174(4), pages 857-879, October.
    18. Frenken, Koen & Hardeman, Sjoerd & Hoekman, Jarno, 2009. "Spatial scientometrics: Towards a cumulative research program," Journal of Informetrics, Elsevier, vol. 3(3), pages 222-232.
    19. Ludo Waltman & Michael Schreiber, 2013. "On the calculation of percentile-based bibliometric indicators," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(2), pages 372-379, February.
    20. González-Pereira, Borja & Guerrero-Bote, Vicente P. & Moya-Anegón, Félix, 2010. "A new approach to the metric of journals’ scientific prestige: The SJR indicator," Journal of Informetrics, Elsevier, vol. 4(3), pages 379-391.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Lutz Bornmann, 2014. "Ranking institutions by the handicap principle," Scientometrics, Springer;Akadémiai Kiadó, vol. 100(2), pages 603-604, August.
    2. Lutz Bornmann & Klaus Wohlrabe, 2019. "Normalisation of citation impact in economics," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(2), pages 841-884, August.
    3. Bornmann, Lutz & Haunschild, Robin & Mutz, Rüdiger, 2020. "Should citations be field-normalized in evaluative bibliometrics? An empirical analysis based on propensity score matching," Journal of Informetrics, Elsevier, vol. 14(4).
    4. Abramo, Giovanni & D’Angelo, Ciriaco Andrea, 2014. "Assessing national strengths and weaknesses in research fields," Journal of Informetrics, Elsevier, vol. 8(3), pages 766-775.
    5. Klaus Wohlrabe & Sabine Gralka & Lutz Bornmann, 2019. "Zur Effizienz deutscher Universitäten und deren Entwicklung zwischen 2004 und 2015," ifo Schnelldienst, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, vol. 72(21), pages 15-21, November.
    6. Lutz Bornmann, 2015. "Alternative metrics in scientometrics: a meta-analysis of research into three altmetrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(3), pages 1123-1144, June.
    7. Xingchen Li & Qiang Wu & Yuanyuan Liu, 2017. "A quantitative analysis of researcher citation personal display considering disciplinary differences and influence factors," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 1093-1112, November.
    8. Klaus Wohlrabe & Elisabeth Friedrich, 2017. "The efficiency of economics departments reconsidered," Economics Bulletin, AccessEcon, vol. 37(3), pages 1602-1611.
    9. Copiello, Sergio, 2019. "Peer and neighborhood effects: Citation analysis using a spatial autoregressive model and pseudo-spatial data," Journal of Informetrics, Elsevier, vol. 13(1), pages 238-254.
    10. Zaida Chinchilla-Rodríguez & Grisel Zacca-González & Benjamín Vargas-Quesada & Félix Moya-Anegón, 2016. "Benchmarking scientific performance by decomposing leadership of Cuban and Latin American institutions in Public Health," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(3), pages 1239-1264, March.
    11. Lutz Bornmann & Rüdiger Mutz & Robin Haunschild & Felix Moya-Anegon & Mirko Almeida Madeira Clemente & Moritz Stefaner, 2021. "Mapping the impact of papers on various status groups in excellencemapping.net: a new release of the excellence mapping tool based on citation and reader scores," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(11), pages 9305-9331, November.
    12. Frenken, Koen & Heimeriks, Gaston J. & Hoekman, Jarno, 2017. "What drives university research performance? An analysis using the CWTS Leiden Ranking data," Journal of Informetrics, Elsevier, vol. 11(3), pages 859-872.
    13. Zsolt Kohus & Márton Demeter & Gyula Péter Szigeti & László Kun & Eszter Lukács & Katalin Czakó, 2022. "The Influence of International Collaboration on the Scientific Impact in V4 Countries," Publications, MDPI, vol. 10(4), pages 1-13, September.
    14. Klaus Wohlrabe & Félix de Moya Anegon & Lutz Bornmann, 2019. "How Efficiently Do Elite US Universities Produce Highly Cited Papers?," Publications, MDPI, vol. 7(1), pages 1-15, January.
    15. Martorell Cunil, Onofre & Otero González, Luis & Durán Santomil, Pablo & Mulet Forteza, Carlos, 2023. "How to accomplish a highly cited paper in the tourism, leisure and hospitality field," Journal of Business Research, Elsevier, vol. 157(C).
    16. Bornmann, Lutz & Williams, Richard, 2017. "Can the journal impact factor be used as a criterion for the selection of junior researchers? A large-scale empirical study based on ResearcherID data," Journal of Informetrics, Elsevier, vol. 11(3), pages 788-799.
    17. Barbara S. Lancho-Barrantes & Hector G. Ceballos-Cancino & Francisco J. Cantu-Ortiz, 2021. "Comparing the efficiency of countries to assimilate and apply research investment," Quality & Quantity: International Journal of Methodology, Springer, vol. 55(4), pages 1347-1369, August.
    18. Marcin Kozak & Lutz Bornmann & Loet Leydesdorff, 2015. "How have the Eastern European countries of the former Warsaw Pact developed since 1990? A bibliometric study," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(2), pages 1101-1117, February.
    19. Gangan Prathap, 2017. "A three-dimensional bibliometric evaluation of recent research in India," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(3), pages 1085-1097, March.
    20. Bornmann, Lutz & Stefaner, Moritz & de Moya Anegón, Felix & Mutz, Rüdiger, 2016. "Excellence networks in science: A Web-based application based on Bayesian multilevel logistic regression (BMLR) for the identification of institutions collaborating successfully," Journal of Informetrics, Elsevier, vol. 10(1), pages 312-327.
    21. Hiran H. Lathabai & Abhirup Nandy & Vivek Kumar Singh, 2021. "x-index: Identifying core competency and thematic research strengths of institutions using an NLP and network based ranking framework," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(12), pages 9557-9583, December.
    22. Tahamtan, Iman & Bornmann, Lutz, 2018. "Core elements in the process of citing publications: Conceptual overview of the literature," Journal of Informetrics, Elsevier, vol. 12(1), pages 203-216.
    23. Klaus Wohlrabe & Felix de Moya Anegon & Lutz Bornmann, 2018. "How efficiently produce elite US universities highly cited papers? A case study based on input and output data," ifo Working Paper Series 264, ifo Institute - Leibniz Institute for Economic Research at the University of Munich.
    24. Aparna Basu & Sumit Kumar Banshal & Khushboo Singhal & Vivek Kumar Singh, 2016. "Designing a Composite Index for research performance evaluation at the national or regional level: ranking Central Universities in India," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(3), pages 1171-1193, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Bornmann, Lutz & Stefaner, Moritz & de Moya Anegón, Felix & Mutz, Rüdiger, 2016. "Excellence networks in science: A Web-based application based on Bayesian multilevel logistic regression (BMLR) for the identification of institutions collaborating successfully," Journal of Informetrics, Elsevier, vol. 10(1), pages 312-327.
    2. Lutz Bornmann & Rüdiger Mutz & Robin Haunschild & Felix Moya-Anegon & Mirko Almeida Madeira Clemente & Moritz Stefaner, 2021. "Mapping the impact of papers on various status groups in excellencemapping.net: a new release of the excellence mapping tool based on citation and reader scores," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(11), pages 9305-9331, November.
    3. Bornmann, Lutz & Leydesdorff, Loet, 2012. "Which are the best performing regions in information science in terms of highly cited papers? Some improvements of our previous mapping approaches," Journal of Informetrics, Elsevier, vol. 6(2), pages 336-345.
    4. Csomós, György, 2018. "Reprint of “A spatial scientometric analysis of the publication output of cities worldwide”," Journal of Informetrics, Elsevier, vol. 12(2), pages 547-566.
    5. Bornmann, Lutz & Leydesdorff, Loet & Wang, Jian, 2014. "How to improve the prediction based on citation impact percentiles for years shortly after the publication date?," Journal of Informetrics, Elsevier, vol. 8(1), pages 175-180.
    6. Mingers, John & Leydesdorff, Loet, 2015. "A review of theory and practice in scientometrics," European Journal of Operational Research, Elsevier, vol. 246(1), pages 1-19.
    7. Abramo, Giovanni & D’Angelo, Ciriaco Andrea, 2015. "A methodology to compute the territorial productivity of scientists: The case of Italy," Journal of Informetrics, Elsevier, vol. 9(4), pages 675-685.
    8. Guangyao Zhang & Yuqi Wang & Weixi Xie & Han Du & Chunlin Jiang & Xianwen Wang, 2021. "The open access usage advantage: a temporal and spatial analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 6187-6199, July.
    9. Bornmann, Lutz & Leydesdorff, Loet & Wang, Jian, 2013. "Which percentile-based approach should be preferred for calculating normalized citation impact values? An empirical comparison of five approaches including a newly developed citation-rank approach (P1," Journal of Informetrics, Elsevier, vol. 7(4), pages 933-944.
    10. János József Tóth & Gergő Háló & Manuel Goyanes, 2023. "Beyond views, productivity, and citations: measuring geopolitical differences of scientific impact in communication research," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(10), pages 5705-5729, October.
    11. Thor, Andreas & Marx, Werner & Leydesdorff, Loet & Bornmann, Lutz, 2016. "Introducing CitedReferencesExplorer (CRExplorer): A program for reference publication year spectroscopy with cited references standardization," Journal of Informetrics, Elsevier, vol. 10(2), pages 503-515.
    12. Loet Leydesdorff & Lutz Bornmann, 2012. "Mapping (USPTO) patent data using overlays to Google Maps," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(7), pages 1442-1458, July.
    13. Mingers, John & Yang, Liying, 2017. "Evaluating journal quality: A review of journal citation indicators and ranking in business and management," European Journal of Operational Research, Elsevier, vol. 257(1), pages 323-337.
    14. Lutz Bornmann & Werner Marx & Andreas Barth, 2013. "The Normalization of Citation Counts Based on Classification Systems," Publications, MDPI, vol. 1(2), pages 1-9, August.
    15. Juan Gorraiz & Ursula Ulrych & Wolfgang Glänzel & Wenceslao Arroyo-Machado & Daniel Torres-Salinas, 2022. "Measuring the excellence contribution at the journal level: an alternative to Garfield’s impact factor," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 7229-7251, December.
    16. Bornmann, Lutz & Ozimek, Adam, 2012. "Stata commands for importing bibliometric data and processing author address information," Journal of Informetrics, Elsevier, vol. 6(4), pages 505-512.
    17. Gerson Pech & Catarina Delgado, 2020. "Assessing the publication impact using citation data from both Scopus and WoS databases: an approach validated in 15 research fields," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 909-924, November.
    18. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    19. Copiello, Sergio, 2019. "Peer and neighborhood effects: Citation analysis using a spatial autoregressive model and pseudo-spatial data," Journal of Informetrics, Elsevier, vol. 13(1), pages 238-254.
    20. Brito, Ricardo & Rodríguez-Navarro, Alonso, 2018. "Research assessment by percentile-based double rank analysis," Journal of Informetrics, Elsevier, vol. 12(1), pages 315-329.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:8:y:2014:i:3:p:581-593. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.