IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v130y2025i4d10.1007_s11192-025-05292-4.html
   My bibliography  Save this article

Can altmetric mentions reflect the quality of evidence? A study in Biomedical and Life Sciences

Author

Listed:
  • Pilar Valderrama

    (Universidad Internacional de La Rioja (UNIR))

  • Wenceslao Arroyo-Machado

    (University of Granada)

  • Adela Baca

    (University of Granada)

  • Daniel Torres-Salinas

    (University of Granada)

Abstract

Although the importance of the social impact of research continues to gain recognition, the extent of its utility is still up for debate. The objective of this study is to evaluate whether the altmetric mentions of publications in Biomedical and Life Sciences are associated with the study design, in view of the “pyramid” of scientific evidence. Data were obtained from PubMed and Altmetric.com. A total of 382,882 documents from 2020to 2021 and corresponding to the following study designs were obtained: Consensus Development Conferences & Guidelines, Systematic Reviews and/or Meta-analysis, Clinical Trials, Observational Studies, Reviews, and Case Reports. The indicators considered were Altmetric Attention Score (AAS), News mentions, Twitter mentions, Facebook mentions, Mendeley readers and Dimensions citations. Only documents with at least one mention/citation were retrieved. Reviews made up 53.9% of the output, followed by Case Reports (17%), while the least frequent was Consensus & Guidelines (0.5%). For each indicator, comparison of mentions/citations among the study designs by the Kruskal–Wallis test gave significant differences in all cases. The hierarchical order occupied by the study designs showed some variations for the different indicators. Global analysis of the altmetrics, could reflect the level of the pyramid of scientific evidence: Consensus Development & Guidelines is situated at the top, but it is followed by Clinical Trial rather than Systematic Review/Meta-analysis. The application of altmetrics to Biomedical and Life Sciences can be useful to discriminate between different study designs and would help identify the most appropriate ones for clinical decision-making.

Suggested Citation

  • Pilar Valderrama & Wenceslao Arroyo-Machado & Adela Baca & Daniel Torres-Salinas, 2025. "Can altmetric mentions reflect the quality of evidence? A study in Biomedical and Life Sciences," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(4), pages 2345-2356, April.
    • Handle: RePEc:spr:scient:v:130:y:2025:i:4:d:10.1007_s11192-025-05292-4
    DOI: 10.1007/s11192-025-05292-4
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-025-05292-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-025-05292-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Christian Gumpenberger & Wolfgang Glänzel & Juan Gorraiz, 2016. "The ecstasy and the agony of the altmetric score," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(2), pages 977-982, August.
    2. repec:plo:pone00:0110469 is not listed on IDEAS
    3. Mike Thelwall, 2017. "Are Mendeley reader counts useful impact indicators in all fields?," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(3), pages 1721-1731, December.
    4. Thelwall, Mike, 2018. "Dimensions: A competitor to Scopus and the Web of Science?," Journal of Informetrics, Elsevier, vol. 12(2), pages 430-435.
    5. Diana Hicks & Paul Wouters & Ludo Waltman & Sarah de Rijcke & Ismael Rafols, 2015. "Bibliometrics: The Leiden Manifesto for research metrics," Nature, Nature, vol. 520(7548), pages 429-431, April.
    6. Mojisola Erdt & Aarthy Nagarajan & Sei-Ching Joanna Sin & Yin-Leng Theng, 2016. "Altmetrics: an analysis of the state-of-the-art in measuring research impact on social media," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1117-1166, November.
    7. José Luis Ortega, 2018. "Reliability and accuracy of altmetric providers: a comparison among Altmetric.com, PlumX and Crossref Event Data," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 2123-2138, September.
    8. Wenceslao Arroyo-Machado & Daniel Torres-Salinas & Nicolas Robinson-Garcia, 2021. "Identifying and characterizing social media communities: a socio-semantic network approach to altmetrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(11), pages 9267-9289, November.
    9. repec:plo:pone00:0203117 is not listed on IDEAS
    10. repec:plo:pone00:0120495 is not listed on IDEAS
    11. Bhaskar Mukherjee & Siniša Subotić & Ajay Kumar Chaubey, 2018. "And now for something completely different: the congruence of the Altmetric Attention Score’s structure between different article groups," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(1), pages 253-275, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ortega, José Luis, 2020. "Proposal of composed altmetric indicators based on prevalence and impact dimensions," Journal of Informetrics, Elsevier, vol. 14(4).
    2. Sergio Copiello, 2020. "Other than detecting impact in advance, alternative metrics could act as early warning signs of retractions: tentative findings of a study into the papers retracted by PLoS ONE," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2449-2469, December.
    3. Daniela De Filippo & Fernanda Morillo & Borja González-Albo, 2023. "Measuring the Impact and Influence of Scientific Activity in the Humanities and Social Sciences," Publications, MDPI, vol. 11(2), pages 1-17, June.
    4. Sergio Copiello, 2019. "Research Interest: another undisclosed (and redundant) algorithm by ResearchGate," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(1), pages 351-360, July.
    5. Maryam Moshtagh & Tahereh Jowkar & Maryam Yaghtin & Hajar Sotudeh, 2023. "The moderating effect of altmetrics on the correlations between single and multi-faceted university ranking systems: the case of THE and QS vs. Nature Index and Leiden," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(1), pages 761-781, January.
    6. Wencan Tian & Zhichao Fang & Xianwen Wang & Rodrigo Costas, 2024. "A multi-dimensional analysis of usage counts, Mendeley readership, and citations for journal and conference papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(2), pages 985-1013, February.
    7. Wenceslao Arroyo-Machado & Daniel Torres-Salinas, 2024. "Stranger things: the vanishing of the Altmetric Attention Score values in information and library science," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(10), pages 6287-6300, October.
    8. Christine Meschede & Tobias Siebenlist, 2018. "Cross-metric compatability and inconsistencies of altmetrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 283-297, April.
    9. Yang, Siluo & Zheng, Mengxue & Yu, Yonghao & Wolfram, Dietmar, 2021. "Are Altmetric.com scores effective for research impact evaluation in the social sciences and humanities?," Journal of Informetrics, Elsevier, vol. 15(1).
    10. Ying Guo & Xiantao Xiao, 2022. "Author-level altmetrics for the evaluation of Chinese scholars," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(2), pages 973-990, February.
    11. Mike Thelwall, 2020. "Data in Brief: Can a mega-journal for data be useful?," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(1), pages 697-709, July.
    12. Ou, Guiyan & Zhao, Kang & Zuo, Renxian & Wu, Jiang, 2024. "Effects of research funding on the academic impact and societal visibility of scientific research," Journal of Informetrics, Elsevier, vol. 18(4).
    13. Bhaskar Mukherjee & Siniša Subotić & Ajay Kumar Chaubey, 2018. "And now for something completely different: the congruence of the Altmetric Attention Score’s structure between different article groups," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(1), pages 253-275, January.
    14. Rafael Repiso & Antonio Castillo-Esparcia & Daniel Torres-Salinas, 2019. "Altmetrics, alternative indicators for Web of Science Communication studies journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(2), pages 941-958, May.
    15. Bornmann, Lutz & Haunschild, Robin & Adams, Jonathan, 2019. "Do altmetrics assess societal impact in a comparable way to case studies? An empirical test of the convergent validity of altmetrics based on data from the UK research excellence framework (REF)," Journal of Informetrics, Elsevier, vol. 13(1), pages 325-340.
    16. Łukasz Wiechetek & Zbigniew Pastuszak, 2022. "Academic social networks metrics: an effective indicator for university performance?," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(3), pages 1381-1401, March.
    17. Sergio Copiello & Pietro Bonifaci, 2019. "ResearchGate Score, full-text research items, and full-text reads: a follow-up study," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(2), pages 1255-1262, May.
    18. Latefa Ali Dardas & Malik Sallam & Amanda Woodward & Nadia Sweis & Narjes Sweis & Faleh A. Sawair, 2023. "Evaluating Research Impact Based on Semantic Scholar Highly Influential Citations, Total Citations, and Altmetric Attention Scores: The Quest for Refined Measures Remains Illusive," Publications, MDPI, vol. 11(1), pages 1-16, January.
    19. Sergio Copiello & Pietro Bonifaci, 2018. "A few remarks on ResearchGate score and academic reputation," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(1), pages 301-306, January.
    20. Daniela Filippo & Pablo Sastrón-Toledo, 2023. "Influence of research on open science in the public policy sphere," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(3), pages 1995-2017, March.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:130:y:2025:i:4:d:10.1007_s11192-025-05292-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.