IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v124y2020i3d10.1007_s11192-020-03564-9.html
   My bibliography  Save this article

An extensive analysis of the presence of altmetric data for Web of Science publications across subject fields and research topics

Author

Listed:
  • Zhichao Fang

    (Leiden University)

  • Rodrigo Costas

    (Leiden University
    Stellenbosch University)

  • Wencan Tian

    (Dalian University of Technology)

  • Xianwen Wang

    (Dalian University of Technology)

  • Paul Wouters

    (Leiden University)

Abstract

Sufficient data presence is one of the key preconditions for applying metrics in practice. Based on both Altmetric.com data and Mendeley data collected up to 2019, this paper presents a state-of-the-art analysis of the presence of 12 kinds of altmetric events for nearly 12.3 million Web of Science publications published between 2012 and 2018. Results show that even though an upward trend of data presence can be observed over time, except for Mendeley readers and Twitter mentions, the overall presence of most altmetric data is still low. The majority of altmetric events go to publications in the fields of Biomedical and Health Sciences, Social Sciences and Humanities, and Life and Earth Sciences. As to research topics, the level of attention received by research topics varies across altmetric data, and specific altmetric data show different preferences for research topics, on the basis of which a framework for identifying hot research topics is proposed and applied to detect research topics with higher levels of attention garnered on certain altmetric data source. Twitter mentions and policy document citations were selected as two examples to identify hot research topics of interest of Twitter users and policy-makers, respectively, shedding light on the potential of altmetric data in monitoring research trends of specific social attention.

Suggested Citation

  • Zhichao Fang & Rodrigo Costas & Wencan Tian & Xianwen Wang & Paul Wouters, 2020. "An extensive analysis of the presence of altmetric data for Web of Science publications across subject fields and research topics," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(3), pages 2519-2549, September.
    • Handle: RePEc:spr:scient:v:124:y:2020:i:3:d:10.1007_s11192-020-03564-9
    DOI: 10.1007/s11192-020-03564-9
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-020-03564-9
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-020-03564-9?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ludo Waltman & Rodrigo Costas, 2014. "F1000 Recommendations as a Potential New Data Source for Research Evaluation: A Comparison With Citations," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(3), pages 433-445, March.
    2. Xianwen Wang & Zhi Wang & Shenmeng Xu, 2013. "Tracing scientist’s research trends realtimely," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(2), pages 717-729, May.
    3. Lutz Bornmann, 2016. "What do altmetrics counts mean? A plea for content analyses," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 67(4), pages 1016-1017, April.
    4. Haunschild, Robin & Leydesdorff, Loet & Bornmann, Lutz & Hellsten, Iina & Marx, Werner, 2019. "Does the public discuss other topics on climate change than researchers? A comparison of explorative networks based on author keywords and hashtags," Journal of Informetrics, Elsevier, vol. 13(2), pages 695-707.
    5. Christine Meschede & Tobias Siebenlist, 2018. "Cross-metric compatability and inconsistencies of altmetrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 283-297, April.
    6. Ludo Waltman & Nees Jan van Eck, 2012. "A new methodology for constructing a publication‐level classification system of science," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(12), pages 2378-2392, December.
    7. Small, Henry & Boyack, Kevin W. & Klavans, Richard, 2014. "Identifying emerging topics in science and technology," Research Policy, Elsevier, vol. 43(8), pages 1450-1467.
    8. Chunmei Gan & Weijun Wang, 2015. "Research characteristics and status on social media in China: A bibliometric and co-word analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(2), pages 1167-1182, November.
    9. Cassidy R. Sugimoto & Sam Work & Vincent Larivière & Stefanie Haustein, 2017. "Scholarly use of social media and altmetrics: A review of the literature," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 68(9), pages 2037-2062, September.
    10. Chaomei Chen, 2006. "CiteSpace II: Detecting and visualizing emerging trends and transient patterns in scientific literature," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 57(3), pages 359-377, February.
    11. Mund, Carolin & Neuhäusler, Peter, 2015. "Towards an early-stage identification of emerging topics in science—The usability of bibliometric characteristics," Journal of Informetrics, Elsevier, vol. 9(4), pages 1018-1033.
    12. Henry Small, 2006. "Tracking and predicting growth areas in science," Scientometrics, Springer;Akadémiai Kiadó, vol. 68(3), pages 595-610, September.
    13. Yuen-Hsien Tseng & Yu-I Lin & Yi-Yang Lee & Wen-Chi Hung & Chun-Hsiang Lee, 2009. "A comparison of methods for detecting hot topics," Scientometrics, Springer;Akadémiai Kiadó, vol. 81(1), pages 73-90, October.
    14. Lutz Bornmann & Robin Haunschild, 2017. "Measuring field-normalized impact of papers on specific societal groups: An altmetrics study based on Mendeley Data," Research Evaluation, Oxford University Press, vol. 26(3), pages 230-241.
    15. Wanying Ding & Chaomei Chen, 2014. "Dynamic topic detection and tracking: A comparison of HDP, C-word, and cocitation methods," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(10), pages 2084-2097, October.
    16. D’Angelo, Ciriaco Andrea & Di Russo, Samuele, 2019. "Testing for universality of Mendeley readership distributions," Journal of Informetrics, Elsevier, vol. 13(2), pages 726-737.
    17. Mojisola Erdt & Aarthy Nagarajan & Sei-Ching Joanna Sin & Yin-Leng Theng, 2016. "Altmetrics: an analysis of the state-of-the-art in measuring research impact on social media," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1117-1166, November.
    18. J. C. F. Winter, 2015. "The relationship between tweets, citations, and article views for PLOS ONE articles," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(2), pages 1773-1779, February.
    19. Ludo Waltman & Nees Jan Eck, 2012. "A new methodology for constructing a publication-level classification system of science," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(12), pages 2378-2392, December.
    20. Werner Marx & Lutz Bornmann, 2015. "On the causes of subject-specific citation rates in Web of Science," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(2), pages 1823-1827, February.
    21. Woo Hyoung Lee, 2008. "How to identify emerging research fields using scientometrics: An example in the field of Information Security," Scientometrics, Springer;Akadémiai Kiadó, vol. 76(3), pages 503-525, September.
    22. Richard Van Noorden, 2014. "Online collaboration: Scientists and the social network," Nature, Nature, vol. 512(7513), pages 126-129, August.
    23. Per O. Seglen, 1992. "The skewness of science," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 43(9), pages 628-638, October.
    24. Lutz Bornmann, 2015. "Alternative metrics in scientometrics: a meta-analysis of research into three altmetrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(3), pages 1123-1144, June.
    25. Zahedi, Zohreh & Haustein, Stefanie, 2018. "On the relationships between bibliographic characteristics of scientific documents and citation and Mendeley readership counts: A large-scale analysis of Web of Science publications," Journal of Informetrics, Elsevier, vol. 12(1), pages 191-202.
    26. Han Zheng & Htet Htet Aung & Mojisola Erdt & Tai‐Quan Peng & Aravind Sesagiri Raamkumar & Yin‐Leng Theng, 2019. "Social media presence of scholarly journals," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 70(3), pages 256-270, March.
    27. Zhichao Fang & Jonathan Dudek & Rodrigo Costas, 2020. "The stability of Twitter metrics: A study on unavailable Twitter mentions of scientific publications," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 71(12), pages 1455-1469, December.
    28. Ehsan Mohammadi & Mike Thelwall & Stefanie Haustein & Vincent Larivière, 2015. "Who reads research articles? An altmetrics analysis of Mendeley user categories," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(9), pages 1832-1846, September.
    29. Rodrigo Costas & Zohreh Zahedi & Paul Wouters, 2015. "Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(10), pages 2003-2019, October.
    30. Zohreh Zahedi & Rodrigo Costas & Paul Wouters, 2017. "Mendeley readership as a filtering tool to identify highly cited publications," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 68(10), pages 2511-2521, October.
    31. Björn Hammarfelt, 2014. "Using altmetrics for assessing research impact in the humanities," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1419-1430, November.
    32. Robin Haunschild & Lutz Bornmann, 2017. "How many scientific papers are mentioned in policy-related documents? An empirical investigation using Web of Science and Altmetric data," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(3), pages 1209-1216, March.
    33. Bornmann, Lutz, 2014. "Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics," Journal of Informetrics, Elsevier, vol. 8(4), pages 895-903.
    34. Zhichao Fang & Rodrigo Costas, 2020. "Studying the accumulation velocity of altmetric data tracked by Altmetric.com," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(2), pages 1077-1101, May.
    35. Fereshteh Didegah & Mike Thelwall, 2018. "Co‐saved, co‐tweeted, and co‐cited networks," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 69(8), pages 959-973, August.
    36. Xianwen Wang & Zhichao Fang & Xiaoling Sun, 2016. "Usage patterns of scholarly articles on Web of Science: a study on Web of Science usage count," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 917-926, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. D’Este, Pablo & Robinson-García, Nicolás, 2023. "Interdisciplinary research and the societal visibility of science: The advantages of spanning multiple and distant scientific fields," Research Policy, Elsevier, vol. 52(2).
    2. González-Betancor, Sara M. & Dorta-González, Pablo, 2023. "Does society show differential attention to researchers based on gender and field?," Journal of Informetrics, Elsevier, vol. 17(4).
    3. Yajie Wang & Alesia Zuccala, 2021. "Scholarly book publishers as publicity agents for SSH titles on Twitter," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(6), pages 4817-4840, June.
    4. João Melo Maricato & Bruno Lara Castro Manso, 2022. "Characterization of the communities of attention interacting with scientific papers on Twitter: altmetric analysis of a Brazilian University," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(7), pages 3815-3835, July.
    5. Zhichao Fang & Rodrigo Costas & Paul Wouters, 2022. "User engagement with scholarly tweets of scientific papers: a large-scale and cross-disciplinary analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(8), pages 4523-4546, August.
    6. Daniela De Filippo & Fernanda Morillo & Borja González-Albo, 2023. "Measuring the Impact and Influence of Scientific Activity in the Humanities and Social Sciences," Publications, MDPI, vol. 11(2), pages 1-17, June.
    7. Michael Taylor, 2023. "Slow, slow, quick, quick, slow: five altmetric sources observed over a decade show evolving trends, by research age, attention source maturity and open access status," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(4), pages 2175-2200, April.
    8. Yang, Siluo & Zheng, Mengxue & Yu, Yonghao & Wolfram, Dietmar, 2021. "Are Altmetric.com scores effective for research impact evaluation in the social sciences and humanities?," Journal of Informetrics, Elsevier, vol. 15(1).
    9. Zhichao Fang & Rodrigo Costas & Wencan Tian & Xianwen Wang & Paul Wouters, 2021. "How is science clicked on Twitter? Click metrics for Bitly short links to scientific publications," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 72(7), pages 918-932, July.
    10. Wencan Tian & Yongzhen Wang & Zhigang Hu & Ruonan Cai & Guangyao Zhang & Xianwen Wang, 2024. "Does Granger causality exist between article usage and publication counts? A topic-level time-series evidence from IEEE Xplore," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(6), pages 3285-3302, June.
    11. Daniel Torres-Salinas & Domingo Docampo & Wenceslao Arroyo-Machado & Nicolas Robinson-Garcia, 2024. "The many publics of science: using altmetrics to identify common communication channels by scientific field," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(7), pages 3705-3723, July.
    12. Lutz Bornmann & Klaus Wohlrabe, 2024. "Recent Temporal Dynamics in Economics: Empirical Analyses of Annual Publications in Economic Fields," CESifo Working Paper Series 10881, CESifo.
    13. Enrique Orduña-Malea & Cristina I. Font-Julián, 2022. "Are patents linked on Twitter? A case study of Google patents," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(11), pages 6339-6362, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sergio Copiello, 2020. "Other than detecting impact in advance, alternative metrics could act as early warning signs of retractions: tentative findings of a study into the papers retracted by PLoS ONE," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2449-2469, December.
    2. Lutz Bornmann & Rüdiger Mutz & Robin Haunschild & Felix Moya-Anegon & Mirko Almeida Madeira Clemente & Moritz Stefaner, 2021. "Mapping the impact of papers on various status groups in excellencemapping.net: a new release of the excellence mapping tool based on citation and reader scores," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(11), pages 9305-9331, November.
    3. Hou, Jianhua & Yang, Xiucai, 2020. "Social media-based sleeping beauties: Defining, identifying and features," Journal of Informetrics, Elsevier, vol. 14(2).
    4. S. Ravikumar & Bidyut Bikash Boruah & M. N. Ravikumar, 2022. "Correlation study between citation count and Mendeley readership of the articles of Sri Lankan authors," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(8), pages 4873-4885, August.
    5. Zhichao Fang & Jonathan Dudek & Rodrigo Costas, 2020. "The stability of Twitter metrics: A study on unavailable Twitter mentions of scientific publications," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 71(12), pages 1455-1469, December.
    6. Mojisola Erdt & Aarthy Nagarajan & Sei-Ching Joanna Sin & Yin-Leng Theng, 2016. "Altmetrics: an analysis of the state-of-the-art in measuring research impact on social media," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1117-1166, November.
    7. Wencan Tian & Yongzhen Wang & Zhigang Hu & Ruonan Cai & Guangyao Zhang & Xianwen Wang, 2024. "Does Granger causality exist between article usage and publication counts? A topic-level time-series evidence from IEEE Xplore," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(6), pages 3285-3302, June.
    8. González-Betancor, Sara M. & Dorta-González, Pablo, 2023. "Does society show differential attention to researchers based on gender and field?," Journal of Informetrics, Elsevier, vol. 17(4).
    9. Zhichao Fang & Rodrigo Costas & Paul Wouters, 2022. "User engagement with scholarly tweets of scientific papers: a large-scale and cross-disciplinary analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(8), pages 4523-4546, August.
    10. Zhichao Fang & Rodrigo Costas & Wencan Tian & Xianwen Wang & Paul Wouters, 2021. "How is science clicked on Twitter? Click metrics for Bitly short links to scientific publications," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 72(7), pages 918-932, July.
    11. Ying Guo & Xiantao Xiao, 2022. "Author-level altmetrics for the evaluation of Chinese scholars," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(2), pages 973-990, February.
    12. Jianhua Hou & Xiucai Yang & Yang Zhang, 2023. "The effect of social media knowledge cascade: an analysis of scientific papers diffusion," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(9), pages 5169-5195, September.
    13. Wang, Zhiqi & Chen, Yue & Glänzel, Wolfgang, 2020. "Preprints as accelerator of scholarly communication: An empirical analysis in Mathematics," Journal of Informetrics, Elsevier, vol. 14(4).
    14. Wenceslao Arroyo-Machado & Daniel Torres-Salinas & Nicolas Robinson-Garcia, 2021. "Identifying and characterizing social media communities: a socio-semantic network approach to altmetrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(11), pages 9267-9289, November.
    15. Lutz Bornmann & Robin Haunschild & Vanash M Patel, 2020. "Are papers addressing certain diseases perceived where these diseases are prevalent? The proposal to use Twitter data as social-spatial sensors," PLOS ONE, Public Library of Science, vol. 15(11), pages 1-22, November.
    16. Liwei Zhang & Jue Wang, 2021. "What affects publications’ popularity on Twitter?," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(11), pages 9185-9198, November.
    17. Bornmann, Lutz & Haunschild, Robin & Adams, Jonathan, 2019. "Do altmetrics assess societal impact in a comparable way to case studies? An empirical test of the convergent validity of altmetrics based on data from the UK research excellence framework (REF)," Journal of Informetrics, Elsevier, vol. 13(1), pages 325-340.
    18. Xuan Zhen Liu & Hui Fang, 2017. "What we can learn from tweets linking to research papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(1), pages 349-369, April.
    19. Latefa Ali Dardas & Malik Sallam & Amanda Woodward & Nadia Sweis & Narjes Sweis & Faleh A. Sawair, 2023. "Evaluating Research Impact Based on Semantic Scholar Highly Influential Citations, Total Citations, and Altmetric Attention Scores: The Quest for Refined Measures Remains Illusive," Publications, MDPI, vol. 11(1), pages 1-16, January.
    20. Maryam Moshtagh & Tahereh Jowkar & Maryam Yaghtin & Hajar Sotudeh, 2023. "The moderating effect of altmetrics on the correlations between single and multi-faceted university ranking systems: the case of THE and QS vs. Nature Index and Leiden," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(1), pages 761-781, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:124:y:2020:i:3:d:10.1007_s11192-020-03564-9. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.