IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v126y2021i5d10.1007_s11192-021-03954-7.html
   My bibliography  Save this article

How accurate are Twitter and Facebook altmetrics data? A comparative content analysis

Author

Listed:
  • Houqiang Yu

    (Sun Yat-Sen University)

  • Biegzat Murat

    (Nanjing University of Science and Technology)

  • Longfei Li

    (Nanjing University of Science and Technology)

  • Tingting Xiao

    (Nanjing Library)

Abstract

Data accuracy is essential for reliable and valid altmetrics analysis. Although Twitter and Facebook altmetrics data are widely used for scholarly communication and scientific evaluation, few studies have tapped into their accuracy issue. Based on content analysis of random sample records over two phases, this study has investigated and compared the accuracy of Twitter and Facebook altmetrics data. Major conclusions are drawn as follows. (1) Three error types were identified from the altmetric data provider and six error types were identified from the altmetric data aggregator. Twitter and Facebook have shared most of the error types except for minor differences in the sub-categories. (2) The overall error rate is substantially high, being 17% and 32% for Twitter and Facebook respectively in April, 2019. However, except for publication date error and posting date error, the percentage of the other error types is relatively low (being around 3%). (3) The percentage of error types related to the dynamic nature of Twitter and Facebook is increasing over time, while percentage of error types concerning the bibliographic data is decreasing over time. (4) The error types are either “high seriousness low percentage” or “low seriousness high percentage”, therefore, they would probably not bring significant negative influence. (5) Underlying reasons of these error types are various. They could be attributable to the Twitter (or Facebook) user, Twitter (or Facebook) platform, altmetric database, as well as the third-party data provider. These results suggest that Twitter and Facebook altmetrics data in the Altmetric database are reliable on the whole, although there is still space for further improvement.

Suggested Citation

  • Houqiang Yu & Biegzat Murat & Longfei Li & Tingting Xiao, 2021. "How accurate are Twitter and Facebook altmetrics data? A comparative content analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(5), pages 4437-4463, May.
    • Handle: RePEc:spr:scient:v:126:y:2021:i:5:d:10.1007_s11192-021-03954-7
    DOI: 10.1007/s11192-021-03954-7
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-021-03954-7
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-021-03954-7?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Houqiang Yu, 2017. "Context of altmetrics data matters: an investigation of count type and user category," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(1), pages 267-283, April.
    2. Zhichao Fang & Rodrigo Costas, 2020. "Studying the accumulation velocity of altmetric data tracked by Altmetric.com," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(2), pages 1077-1101, May.
    3. Yu, Houqiang & Xu, Shenmeng & Xiao, Tingting & Hemminger, Brad M. & Yang, Siluo, 2017. "Global science discussed in local altmetrics: Weibo and its comparison with Twitter," Journal of Informetrics, Elsevier, vol. 11(2), pages 466-482.
    4. Bornmann, Lutz, 2014. "Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics," Journal of Informetrics, Elsevier, vol. 8(4), pages 895-903.
    5. Bornmann, Lutz & Haunschild, Robin & Adams, Jonathan, 2019. "Do altmetrics assess societal impact in a comparable way to case studies? An empirical test of the convergent validity of altmetrics based on data from the UK research excellence framework (REF)," Journal of Informetrics, Elsevier, vol. 13(1), pages 325-340.
    6. Zhichao Fang & Jonathan Dudek & Rodrigo Costas, 2020. "The stability of Twitter metrics: A study on unavailable Twitter mentions of scientific publications," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 71(12), pages 1455-1469, December.
    7. Houqiang Yu & Xueting Cao & Tingting Xiao & Zhenyi Yang, 2020. "How accurate are policy document mentions? A first look at the role of altmetrics database," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1517-1540, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Wenceslao Arroyo-Machado & Daniel Torres-Salinas, 2024. "Stranger things: the vanishing of the Altmetric Attention Score values in information and library science," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(10), pages 6287-6300, October.
    2. Houqiang Yu & Xinyun Yu & Xueting Cao, 2022. "How accurate are news mentions of scholarly output? A content analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(7), pages 4075-4096, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. González-Betancor, Sara M. & Dorta-González, Pablo, 2023. "Does society show differential attention to researchers based on gender and field?," Journal of Informetrics, Elsevier, vol. 17(4).
    2. Lin Zhang & Zhenyu Gou & Zhichao Fang & Gunnar Sivertsen & Ying Huang, 2023. "Who tweets scientific publications? A large‐scale study of tweeting audiences in all areas of research," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 74(13), pages 1485-1497, December.
    3. Ying Guo & Xiantao Xiao, 2022. "Author-level altmetrics for the evaluation of Chinese scholars," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(2), pages 973-990, February.
    4. Zhichao Fang & Rodrigo Costas & Wencan Tian & Xianwen Wang & Paul Wouters, 2021. "How is science clicked on Twitter? Click metrics for Bitly short links to scientific publications," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 72(7), pages 918-932, July.
    5. Zhichao Fang & Rodrigo Costas & Wencan Tian & Xianwen Wang & Paul Wouters, 2020. "An extensive analysis of the presence of altmetric data for Web of Science publications across subject fields and research topics," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(3), pages 2519-2549, September.
    6. Sergio Copiello, 2020. "Other than detecting impact in advance, alternative metrics could act as early warning signs of retractions: tentative findings of a study into the papers retracted by PLoS ONE," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2449-2469, December.
    7. Kaltrina Nuredini, 2021. "Investigating Altmetric Information For The Top 1000 Journals From Handelsblatt Ranking In Economic And Business Studies," Journal of Economic Surveys, Wiley Blackwell, vol. 35(5), pages 1315-1343, December.
    8. Siluo Yang & Xin Xing & Dietmar Wolfram, 2018. "Difference in the impact of open-access papers published by China and the USA," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(2), pages 1017-1037, May.
    9. Yu, Houqiang & Xiao, Tingting & Xu, Shenmeng & Wang, Yuefen, 2019. "Who posts scientific tweets? An investigation into the productivity, locations, and identities of scientific tweeters," Journal of Informetrics, Elsevier, vol. 13(3), pages 841-855.
    10. Jianhua Hou & Bili Zheng & Yang Zhang & Chaomei Chen, 2021. "How do Price medalists’ scholarly impact change before and after their awards?," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 5945-5981, July.
    11. Zhang, Lin & Sivertsen, Gunnar & Du, Huiying & HUANG, Ying & Glänzel, Wolfgang, 2021. "Gender differences in the aims and impacts of research," SocArXiv 9n347, Center for Open Science.
    12. Rongying Zhao & Xu Wang, 2019. "Evaluation and comparison of influence in international Open Access journals between China and USA," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(3), pages 1091-1110, September.
    13. Zhichao Fang & Rodrigo Costas & Paul Wouters, 2022. "User engagement with scholarly tweets of scientific papers: a large-scale and cross-disciplinary analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(8), pages 4523-4546, August.
    14. Jianhua Hou & Da Ma, 2020. "How the high-impact papers formed? A study using data from social media and citation," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2597-2615, December.
    15. Zhichao Fang & Rodrigo Costas, 2020. "Studying the accumulation velocity of altmetric data tracked by Altmetric.com," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(2), pages 1077-1101, May.
    16. Yang, Siluo & Zheng, Mengxue & Yu, Yonghao & Wolfram, Dietmar, 2021. "Are Altmetric.com scores effective for research impact evaluation in the social sciences and humanities?," Journal of Informetrics, Elsevier, vol. 15(1).
    17. Yashan Li & Jinge Mao & Lin Zhang & Dongbo Wang & Si Shen & Ying Huang, 2022. "How scientific research incorporates policy: an examination using the case of China’s science and technology evaluation system," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(9), pages 5283-5306, September.
    18. Houqiang Yu & Xinyun Yu & Xueting Cao, 2022. "How accurate are news mentions of scholarly output? A content analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(7), pages 4075-4096, July.
    19. Chieh Liu & Mu-Hsuan Huang, 2022. "Exploring the relationships between altmetric counts and citations of papers in different academic fields based on co-occurrence analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(8), pages 4939-4958, August.
    20. Lutz Bornmann & Robin Haunschild & Vanash M Patel, 2020. "Are papers addressing certain diseases perceived where these diseases are prevalent? The proposal to use Twitter data as social-spatial sensors," PLOS ONE, Public Library of Science, vol. 15(11), pages 1-22, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:126:y:2021:i:5:d:10.1007_s11192-021-03954-7. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.