IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v110y2017i2d10.1007_s11192-016-2200-2.html
   My bibliography  Save this article

Does evaluative scientometrics lose its main focus on scientific quality by the new orientation towards societal impact?

Author

Listed:
  • Lutz Bornmann

    (Administrative Headquarters of the Max Planck Society)

  • Robin Haunschild

    (Max Planck Institute for Solid State Research)

Abstract

When the meaning of key terms is incompatible in competing taxonomies, a revolution might occur in the field by which the established taxonomy is replaced with another. Since the key term “impact” in scientometrics seems to undergo a taxonomic change, a revolution might be taking place at present: Impact is no longer defined as impact on science alone (measured by citations), but on all sectors of society (e.g. economics, culture, or politics). In this Short Communication, we outline that the current revolution in scientometrics does not only imply a broadening of the impact perspective, but also the devaluation of quality considerations in evaluative contexts. Impact might no longer be seen as a proxy for quality, but in its original sense: the simple resonance in some sectors of society.

Suggested Citation

  • Lutz Bornmann & Robin Haunschild, 2017. "Does evaluative scientometrics lose its main focus on scientific quality by the new orientation towards societal impact?," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(2), pages 937-943, February.
    • Handle: RePEc:spr:scient:v:110:y:2017:i:2:d:10.1007_s11192-016-2200-2
    DOI: 10.1007/s11192-016-2200-2
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-016-2200-2
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-016-2200-2?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. M.H. MacRoberts & B.R. MacRoberts, 2010. "Problems of citation analysis: A study of uncited and seldom-cited influences," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(1), pages 1-12, January.
    2. M. H. MacRoberts & B. R. MacRoberts, 1987. "Another test of the normative theory of citing," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 38(4), pages 305-306, July.
    3. M. H. MacRoberts & B. R. MacRoberts, 1988. "Author motivation for not citing influences: A methodological note," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 39(6), pages 432-433, November.
    4. Lutz Bornmann & Robin Haunschild & Werner Marx, 2016. "Policy documents as sources for measuring societal impact: how often is climate change research mentioned in policy-related documents?," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1477-1495, December.
    5. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    6. Bornmann, Lutz & Mutz, Rüdiger & Hug, Sven E. & Daniel, Hans-Dieter, 2011. "A multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants," Journal of Informetrics, Elsevier, vol. 5(3), pages 346-359.
    7. M.H. MacRoberts & B.R. MacRoberts, 2010. "Problems of citation analysis: A study of uncited and seldom‐cited influences," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 61(1), pages 1-12, January.
    8. Nicky J. Welton & Howard H. Z. Thom, 2015. "Value of Information," Medical Decision Making, , vol. 35(5), pages 564-566, July.
    9. M. H. MacRoberts & B. R. MacRoberts, 1997. "Citation content analysis of a botany journal," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 48(3), pages 274-275, March.
    10. Lutz Bornmann, 2014. "Is there currently a scientific revolution in Scientometrics?," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(3), pages 647-648, March.
    11. Lutz Bornmann, 2015. "Alternative metrics in scientometrics: a meta-analysis of research into three altmetrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(3), pages 1123-1144, June.
    12. Franceschet, Massimo & Costantini, Antonio, 2011. "The first Italian research assessment exercise: A bibliometric perspective," Journal of Informetrics, Elsevier, vol. 5(2), pages 275-291.
    13. Lutz Bornmann & Hans-Dieter Daniel, 2005. "Does the h-index for ranking of scientists really work?," Scientometrics, Springer;Akadémiai Kiadó, vol. 65(3), pages 391-392, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Alexander Schniedermann, 2021. "A comparison of systematic reviews and guideline-based systematic reviews in medical studies," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(12), pages 9829-9846, December.
    2. Sergio Copiello & Pietro Bonifaci, 2019. "ResearchGate Score, full-text research items, and full-text reads: a follow-up study," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(2), pages 1255-1262, May.
    3. Brito, Ricardo & Navarro, Alonso Rodríguez, 2021. "The inconsistency of h-index: A mathematical analysis," Journal of Informetrics, Elsevier, vol. 15(1).
    4. Lutz Bornmann & Robin Haunschild, 2017. "Quality and impact considerations in bibliometrics: a reply to Ricker (in press)," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(3), pages 1857-1859, June.
    5. Yashan Li & Jinge Mao & Lin Zhang & Dongbo Wang & Si Shen & Ying Huang, 2022. "How scientific research incorporates policy: an examination using the case of China’s science and technology evaluation system," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(9), pages 5283-5306, September.
    6. Liwei Zhang & Jue Wang, 2018. "Why highly cited articles are not highly tweeted? A biology case," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(1), pages 495-509, October.
    7. Drahomira Herrmannova & Robert M. Patton & Petr Knoth & Christopher G. Stahl, 2018. "Do citations and readership identify seminal publications?," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 239-262, April.
    8. Tanel Hirv, 2022. "The interplay of the size of the research system, ways of collaboration, level, and method of funding in determining bibliometric outputs," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(3), pages 1295-1316, March.
    9. Martin Ricker, 2017. "Letter to the Editor: About the quality and impact of scientific articles," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(3), pages 1851-1855, June.
    10. Sergio Copiello, 2020. "Other than detecting impact in advance, alternative metrics could act as early warning signs of retractions: tentative findings of a study into the papers retracted by PLoS ONE," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2449-2469, December.
    11. Xiaozan Lyu & Rodrigo Costas, 2020. "How do academic topics shift across altmetric sources? A case study of the research area of Big Data," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(2), pages 909-943, May.
    12. de Carvalho, Gustavo Dambiski Gomes & Sokulski, Carla Cristiane & da Silva, Wesley Vieira & de Carvalho, Hélio Gomes & de Moura, Rafael Vignoli & de Francisco, Antonio Carlos & da Veiga, Claudimar Per, 2020. "Bibliometrics and systematic reviews: A comparison between the Proknow-C and the Methodi Ordinatio," Journal of Informetrics, Elsevier, vol. 14(3).
    13. Bornmann, Lutz & Adams, Jonathan & Leydesdorff, Loet, 2018. "The negative effects of citing with a national orientation in terms of recognition: National and international citations in natural-sciences papers from Germany, the Netherlands, and the UK," Journal of Informetrics, Elsevier, vol. 12(3), pages 931-949.
    14. Kuklin, Alexander A. (Куклин, Александр) & Balyakina, Evgeniya A. (Балякина, Евгения), 2017. "Active policy as a key to success for an International Economic Periodical [Активная Политика — Залог Успеха Международного Экономического Журнала]," Ekonomicheskaya Politika / Economic Policy, Russian Presidential Academy of National Economy and Public Administration, vol. 6, pages 160-177, December.
    15. Sergio Copiello & Pietro Bonifaci, 2018. "A few remarks on ResearchGate score and academic reputation," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(1), pages 301-306, January.
    16. Dotti, Nicola Francesco & Walczyk, Julia, 2022. "What is the societal impact of university research? A policy-oriented review to map approaches, identify monitoring methods and success factors," Evaluation and Program Planning, Elsevier, vol. 95(C).
    17. Maureen McKelvey & Bastian Rake, 2020. "Exploring scientific publications by firms: what are the roles of academic and corporate partners for publications in high reputation or high impact journals?," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(3), pages 1323-1360, March.
    18. Lutz Bornmann & Robin Haunschild & Vanash M Patel, 2020. "Are papers addressing certain diseases perceived where these diseases are prevalent? The proposal to use Twitter data as social-spatial sensors," PLOS ONE, Public Library of Science, vol. 15(11), pages 1-22, November.
    19. Yang, Siluo & Zheng, Mengxue & Yu, Yonghao & Wolfram, Dietmar, 2021. "Are Altmetric.com scores effective for research impact evaluation in the social sciences and humanities?," Journal of Informetrics, Elsevier, vol. 15(1).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Abramo, Giovanni, 2018. "Revisiting the scientometric conceptualization of impact and its measurement," Journal of Informetrics, Elsevier, vol. 12(3), pages 590-597.
    2. Bornmann, Lutz & Haunschild, Robin & Adams, Jonathan, 2019. "Do altmetrics assess societal impact in a comparable way to case studies? An empirical test of the convergent validity of altmetrics based on data from the UK research excellence framework (REF)," Journal of Informetrics, Elsevier, vol. 13(1), pages 325-340.
    3. Tobias Opthof & Loet Leydesdorff, 2011. "A comment to the paper by Waltman et al., Scientometrics, 87, 467–481, 2011," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(3), pages 1011-1016, September.
    4. Bornmann, Lutz & Haunschild, Robin & Mutz, Rüdiger, 2020. "Should citations be field-normalized in evaluative bibliometrics? An empirical analysis based on propensity score matching," Journal of Informetrics, Elsevier, vol. 14(4).
    5. Jiang, Xiaorui & Zhuge, Hai, 2019. "Forward search path count as an alternative indirect citation impact indicator," Journal of Informetrics, Elsevier, vol. 13(4).
    6. Mike Thelwall, 2019. "The influence of highly cited papers on field normalised indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(2), pages 519-537, February.
    7. Ehsan Mohammadi & Mike Thelwall, 2013. "Assessing non-standard article impact using F1000 labels," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(2), pages 383-395, November.
    8. Tahamtan, Iman & Bornmann, Lutz, 2018. "Creativity in science and the link to cited references: Is the creative potential of papers reflected in their cited references?," Journal of Informetrics, Elsevier, vol. 12(3), pages 906-930.
    9. Drahomira Herrmannova & Robert M. Patton & Petr Knoth & Christopher G. Stahl, 2018. "Do citations and readership identify seminal publications?," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 239-262, April.
    10. Bornmann, Lutz & Haunschild, Robin, 2018. "Normalization of zero-inflated data: An empirical analysis of a new indicator family and its use with altmetrics data," Journal of Informetrics, Elsevier, vol. 12(3), pages 998-1011.
    11. Bornmann, Lutz & Marx, Werner, 2018. "Critical rationalism and the search for standard (field-normalized) indicators in bibliometrics," Journal of Informetrics, Elsevier, vol. 12(3), pages 598-604.
    12. Thelwall, Mike & Nevill, Tamara, 2018. "Could scientists use Altmetric.com scores to predict longer term citation counts?," Journal of Informetrics, Elsevier, vol. 12(1), pages 237-248.
    13. Ziqiang Zeng & Lantian Shi, 2021. "A two-dimensional journal classification method based on output and input factors: perspectives from citation and authorship related indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(5), pages 3929-3964, May.
    14. Sten F Odenwald, 2020. "A citation study of earth science projects in citizen science," PLOS ONE, Public Library of Science, vol. 15(7), pages 1-26, July.
    15. Martin Ricker, 2017. "Letter to the Editor: About the quality and impact of scientific articles," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(3), pages 1851-1855, June.
    16. Zhenbin Yan & Qiang Wu & Xingchen Li, 2016. "Do Hirsch-type indices behave the same in assessing single publications? An empirical study of 29 bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1815-1833, December.
    17. Thelwall, Mike, 2018. "Dimensions: A competitor to Scopus and the Web of Science?," Journal of Informetrics, Elsevier, vol. 12(2), pages 430-435.
    18. Yang, Siluo & Han, Ruizhen & Wolfram, Dietmar & Zhao, Yuehua, 2016. "Visualizing the intellectual structure of information science (2006–2015): Introducing author keyword coupling analysis," Journal of Informetrics, Elsevier, vol. 10(1), pages 132-150.
    19. Diana Hicks & Julia Melkers & Kimberley R. Isett, 2019. "A characterization of professional media and its links to research," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(2), pages 827-843, May.
    20. Parul Khurana & Kiran Sharma, 2022. "Impact of h-index on author’s rankings: an improvement to the h-index for lower-ranked authors," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(8), pages 4483-4498, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:110:y:2017:i:2:d:10.1007_s11192-016-2200-2. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.