IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v101y2014i2d10.1007_s11192-014-1411-7.html
   My bibliography  Save this article

Measuring the scientific impact of e-research infrastructures: a citation based approach?

Author

Listed:
  • K. Jonkers

    (CSIC Institute for Public Goods and Policies)

  • G. E. Derrick

    (CSIC Institute for Public Goods and Policies
    Brunel University)

  • C. Lopez-Illescas

    (University of Extremadura
    Vrije University van Amsterdam)

  • P. Besselaar

    (Vrije University van Amsterdam)

Abstract

This micro-level study explores the extent that citation analysis provides an accurate and representative assessment of the use and impact of bioinformatics e-research infrastructure. The bioinformatic e-research infrastructure studied offers common tools used by life scientists to analyse and interpret genetic and protein sequence information. These e-resources therefore provide an interesting example with which to explore how representative citations are as acknowledgements of knowledge in the life sciences. The examples presented here suggest that there is a relation between number of visits to these databases and number of citations; however, a parallel finding shows how citation analysis frequently underestimates acknowledged use of the resources offered on this e-research infrastructure. The paper discusses the implications of the findings for various aspects of impact measurement and also considers how appropriate citation analysis is as a measurement of knowledge claims.

Suggested Citation

  • K. Jonkers & G. E. Derrick & C. Lopez-Illescas & P. Besselaar, 2014. "Measuring the scientific impact of e-research infrastructures: a citation based approach?," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1179-1194, November.
    • Handle: RePEc:spr:scient:v:101:y:2014:i:2:d:10.1007_s11192-014-1411-7
    DOI: 10.1007/s11192-014-1411-7
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-014-1411-7
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-014-1411-7?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. López-Illescas, Carmen & de Moya-Anegón, Félix & Moed, Henk F., 2008. "Coverage and citation impact of oncological journals in the Web of Science and Scopus," Journal of Informetrics, Elsevier, vol. 2(4), pages 304-316.
    2. Martin, Ben R. & Irvine, John, 1993. "Assessing basic research : Some partial indicators of scientific progress in radio astronomy," Research Policy, Elsevier, vol. 22(2), pages 106-106, April.
    3. Koen Jonkers & Felix de Moya Anegon & Isidro F. Aguillo, 2012. "Measuring the usage of e‐research infrastructure as an indicator of research activity," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(7), pages 1374-1382, July.
    4. Senker, Jacqueline, 1995. "Tacit Knowledge and Models of Innovation," Industrial and Corporate Change, Oxford University Press and the Associazione ICC, vol. 4(2), pages 425-447.
    5. Norman Kaplan, 1965. "The norms of citation behavior: Prolegomena to the footnote," American Documentation, Wiley Blackwell, vol. 16(3), pages 179-184, July.
    6. Peter Weingart, 2005. "Impact of bibliometrics upon the science system: Inadvertent consequences?," Scientometrics, Springer;Akadémiai Kiadó, vol. 62(1), pages 117-131, January.
    7. Stefan P L de Jong & Pleun van Arensbergen & Floortje Daemen & Barend van der Meulen & Peter van den Besselaar, 2011. "Evaluation of research in context: an approach and two cases," Research Evaluation, Oxford University Press, vol. 20(1), pages 61-72, March.
    8. Félix Moya-Anegón & Zaida Chinchilla-Rodríguez & Benjamín Vargas-Quesada & Elena Corera-Álvarez & Francisco José Muñoz-Fernández & Antonio González-Molina & Victor Herrero-Solana, 2007. "Coverage analysis of Scopus: A journal metric approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 73(1), pages 53-78, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Rafols, Ismael & Leydesdorff, Loet & O’Hare, Alice & Nightingale, Paul & Stirling, Andy, 2012. "How journal rankings can suppress interdisciplinary research: A comparison between Innovation Studies and Business & Management," Research Policy, Elsevier, vol. 41(7), pages 1262-1282.
    2. Aksnes, Dag W. & Rip, Arie, 2009. "Researchers' perceptions of citations," Research Policy, Elsevier, vol. 38(6), pages 895-905, July.
    3. Bornmann, Lutz & Leydesdorff, Loet, 2015. "Does quality and content matter for citedness? A comparison with para-textual factors and over time," Journal of Informetrics, Elsevier, vol. 9(3), pages 419-429.
    4. Maaike Verbree & Edwin Horlings & Peter Groenewegen & Inge Weijden & Peter Besselaar, 2015. "Organizational factors influencing scholarly performance: a multivariate study of biomedical research groups," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 25-49, January.
    5. Maxim N. Kotsemir & Tatiana E. Kuznetsova & Elena G. Nasybulina & Anna G. Pikalova, 2015. "Empirical Analysis of Multinational S&T Collaboration Priorities –The Case of Russia," HSE Working papers WP BRP 53/STI/2015, National Research University Higher School of Economics.
    6. Lutz Bornmann & Julian N. Marewski, 2019. "Heuristics as conceptual lens for understanding and studying the usage of bibliometrics in research evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(2), pages 419-459, August.
    7. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    8. Abramo, Giovanni, 2018. "Revisiting the scientometric conceptualization of impact and its measurement," Journal of Informetrics, Elsevier, vol. 12(3), pages 590-597.
    9. Thelwall, Mike & Sud, Pardeep, 2016. "National, disciplinary and temporal variations in the extent to which articles with more authors have more impact: Evidence from a geometric field normalised citation indicator," Journal of Informetrics, Elsevier, vol. 10(1), pages 48-61.
    10. A. Abrizah & A. N. Zainab & K. Kiran & R. G. Raj, 2013. "LIS journals scientific impact and subject categorization: a comparison between Web of Science and Scopus," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(2), pages 721-740, February.
    11. López-Illescas, Carmen & de Moya-Anegón, Félix & Moed, Henk F., 2008. "Coverage and citation impact of oncological journals in the Web of Science and Scopus," Journal of Informetrics, Elsevier, vol. 2(4), pages 304-316.
    12. Emanuela Reale & Anna Barbara & Antonio Costantini, 2006. "Peer review for the evaluation of the academic research: the Italian experience," CERIS Working Paper 200615, CNR-IRCrES Research Institute on Sustainable Economic Growth - Torino (TO) ITALY - former Institute for Economic Research on Firms and Growth - Moncalieri (TO) ITALY.
    13. Berlemann, Michael & Haucap, Justus, 2015. "Which factors drive the decision to opt out of individual research rankings? An empirical study of academic resistance to change," Research Policy, Elsevier, vol. 44(5), pages 1108-1115.
    14. Fairclough, Ruth & Thelwall, Mike, 2015. "More precise methods for national research citation impact comparisons," Journal of Informetrics, Elsevier, vol. 9(4), pages 895-906.
    15. Maja Jokic & Josip Stepanic & Boris Kamenar & Vlatko Silobrcic, 2006. "Research Output of Croatian Universities from 1996 to 2004, Registered by the Science Citation Index-Expanded," Interdisciplinary Description of Complex Systems - scientific journal, Croatian Interdisciplinary Society Provider Homepage: http://indecs.eu, vol. 4(1), pages 44-50.
    16. Abramo, Giovanni & D’Angelo, Ciriaco Andrea & Di Costa, Flavia, 2021. "On the relation between the degree of internationalization of cited and citing publications: A field level analysis, including and excluding self-citations," Journal of Informetrics, Elsevier, vol. 15(1).
    17. Jarle Hildrum & Dieter Ernst & Jan Fagerberg, 2011. "The Complex Interaction between Global Production Networks, Digital Information Systems and International Knowledge Transfers," Chapters, in: Cristiano Antonelli (ed.), Handbook on the Economic Complexity of Technological Change, chapter 16, Edward Elgar Publishing.
    18. Müller, Harry, 2012. "Die Zitationshäufigkeit als Qualitätsindikator im Rahmen der Forschungsleistungsmessung," Discussion Papers of the Institute for Organisational Economics 1/2012, University of Münster, Institute for Organisational Economics.
    19. Ricardo Arencibia-Jorge & Felix Moya-Anegón, 2010. "Challenges in the study of Cuban scientific output," Scientometrics, Springer;Akadémiai Kiadó, vol. 83(3), pages 723-737, June.
    20. Beck, Mathias & Junge, Martin & Kaiser, Ulrich, 2017. "Public Funding and Corporate Innovation," IZA Discussion Papers 11196, Institute of Labor Economics (IZA).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:101:y:2014:i:2:d:10.1007_s11192-014-1411-7. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.