IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v8y2014i4p895-903.html
   My bibliography  Save this article

Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics

Author

Listed:
  • Bornmann, Lutz

Abstract

Today, it is not clear how the impact of research on other areas of society than science should be measured. While peer review and bibliometrics have become standard methods for measuring the impact of research in science, there is not yet an accepted framework within which to measure societal impact. Alternative metrics (called altmetrics to distinguish them from bibliometrics) are considered an interesting option for assessing the societal impact of research, as they offer new ways to measure (public) engagement with research output. Altmetrics is a term to describe web-based metrics for the impact of publications and other scholarly material by using data from social media platforms (e.g. Twitter or Mendeley). This overview of studies explores the potential of altmetrics for measuring societal impact. It deals with the definition and classification of altmetrics. Furthermore, their benefits and disadvantages for measuring impact are discussed.

Suggested Citation

  • Bornmann, Lutz, 2014. "Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics," Journal of Informetrics, Elsevier, vol. 8(4), pages 895-903.
    • Handle: RePEc:eee:infome:v:8:y:2014:i:4:p:895-903
    DOI: 10.1016/j.joi.2014.09.005
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157714000868
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.joi.2014.09.005?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Liwen Vaughan & Debora Shaw, 2005. "Web citation data for impact assessment: A comparison of four science disciplines," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 56(10), pages 1075-1087, August.
    2. Kim Holmberg & Mike Thelwall, 2014. "Disciplinary differences in Twitter scholarly communication," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1027-1042, November.
    3. Lutz Bornmann, 2013. "What is societal impact of research and how can it be assessed? a literature survey," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(2), pages 217-233, February.
    4. Bornmann, Lutz & Leydesdorff, Loet & Mutz, Rüdiger, 2013. "The use of percentiles and percentile rank classes in the analysis of bibliometric data: Opportunities and limits," Journal of Informetrics, Elsevier, vol. 7(1), pages 158-165.
    5. Cameron Neylon & Shirley Wu, 2009. "Article-Level Metrics and the Evolution of Scientific Impact," PLOS Biology, Public Library of Science, vol. 7(11), pages 1-6, November.
    6. Bornmann, Lutz & Williams, Richard, 2013. "How to calculate the practical significance of citation impact differences? An empirical example from evaluative institutional bibliometrics using adjusted predictions and marginal effects," Journal of Informetrics, Elsevier, vol. 7(2), pages 562-574.
    7. Xuemei Li & Mike Thelwall & Dean Giustini, 2012. "Validating online reference managers for scholarly impact measurement," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(2), pages 461-471, May.
    8. Stefanie Haustein & Isabella Peters & Judit Bar-Ilan & Jason Priem & Hadas Shema & Jens Terliesner, 2014. "Coverage and adoption of altmetrics sources in the bibliometric community," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1145-1163, November.
    9. Stefanie Haustein & Isabella Peters & Cassidy R. Sugimoto & Mike Thelwall & Vincent Larivière, 2014. "Tweeting biomedicine: An analysis of tweets and citations in the biomedical literature," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(4), pages 656-669, April.
    10. Blaise Cronin, 2013. "The evolving indicator space (iSpace)," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(8), pages 1523-1525, August.
    11. Jian Wang, 2013. "Citation time window choice for research impact evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(3), pages 851-872, March.
    12. Liwen Vaughan & Debora Shaw, 2008. "A new look at evidence of scholarly citation in citation indexes and from web sources," Scientometrics, Springer;Akadémiai Kiadó, vol. 74(2), pages 317-330, February.
    13. Heather Piwowar, 2013. "Value all research products," Nature, Nature, vol. 493(7431), pages 159-159, January.
    14. King, Dominic & Ramirez-Cano, Daniel & Greaves, Felix & Vlaev, Ivo & Beales, Steve & Darzi, Ara, 2013. "Twitter and the health reforms in the English National Health Service," Health Policy, Elsevier, vol. 110(2), pages 291-297.
    15. Ludo Waltman & Rodrigo Costas, 2014. "F1000 Recommendations as a Potential New Data Source for Research Evaluation: A Comparison With Citations," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(3), pages 433-445, March.
    16. Blaise Cronin, 2013. "The evolving indicator space (iSpace)," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(8), pages 1523-1525, August.
    17. Ehsan Mohammadi & Mike Thelwall, 2013. "Assessing non-standard article impact using F1000 labels," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(2), pages 383-395, November.
    18. Lutz Bornmann, 2014. "Is there currently a scientific revolution in Scientometrics?," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(3), pages 647-648, March.
    19. Ehsan Mohammadi & Mike Thelwall, 2014. "Mendeley readership altmetrics for the social sciences and humanities: Research evaluation and knowledge flows," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(8), pages 1627-1638, August.
    20. Hadas Shema & Judit Bar-Ilan & Mike Thelwall, 2014. "Do blog citations correlate with a higher number of future citations? Research blogs as a potential source for alternative metrics," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(5), pages 1018-1027, May.
    21. Björn Hammarfelt, 2014. "Using altmetrics for assessing research impact in the humanities," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1419-1430, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mojisola Erdt & Aarthy Nagarajan & Sei-Ching Joanna Sin & Yin-Leng Theng, 2016. "Altmetrics: an analysis of the state-of-the-art in measuring research impact on social media," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1117-1166, November.
    2. Lutz Bornmann, 2015. "Alternative metrics in scientometrics: a meta-analysis of research into three altmetrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(3), pages 1123-1144, June.
    3. Ehsan Mohammadi & Mike Thelwall & Stefanie Haustein & Vincent Larivière, 2015. "Who reads research articles? An altmetrics analysis of Mendeley user categories," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(9), pages 1832-1846, September.
    4. Amalia Mas-Bleda & Mike Thelwall, 2016. "Can alternative indicators overcome language biases in citation counts? A comparison of Spanish and UK research," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 2007-2030, December.
    5. Bornmann, Lutz, 2014. "Validity of altmetrics data for measuring societal impact: A study using data from Altmetric and F1000Prime," Journal of Informetrics, Elsevier, vol. 8(4), pages 935-950.
    6. Bornmann, Lutz & Haunschild, Robin & Adams, Jonathan, 2019. "Do altmetrics assess societal impact in a comparable way to case studies? An empirical test of the convergent validity of altmetrics based on data from the UK research excellence framework (REF)," Journal of Informetrics, Elsevier, vol. 13(1), pages 325-340.
    7. Liwei Zhang & Jue Wang, 2018. "Why highly cited articles are not highly tweeted? A biology case," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(1), pages 495-509, October.
    8. Sergio Copiello, 2020. "Other than detecting impact in advance, alternative metrics could act as early warning signs of retractions: tentative findings of a study into the papers retracted by PLoS ONE," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2449-2469, December.
    9. Jianhua Hou & Xiucai Yang & Yang Zhang, 2023. "The effect of social media knowledge cascade: an analysis of scientific papers diffusion," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(9), pages 5169-5195, September.
    10. Isidro F. Aguillo, 2020. "Altmetrics of the Open Access Institutional Repositories: a webometrics approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(3), pages 1181-1192, June.
    11. Yu Liu & Dan Lin & Xiujuan Xu & Shimin Shan & Quan Z. Sheng, 2018. "Multi-views on Nature Index of Chinese academic institutions," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 823-837, March.
    12. Maryam Moshtagh & Tahereh Jowkar & Maryam Yaghtin & Hajar Sotudeh, 2023. "The moderating effect of altmetrics on the correlations between single and multi-faceted university ranking systems: the case of THE and QS vs. Nature Index and Leiden," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(1), pages 761-781, January.
    13. Thelwall, Mike & Wilson, Paul, 2014. "Regression for citation data: An evaluation of different methods," Journal of Informetrics, Elsevier, vol. 8(4), pages 963-971.
    14. Yu, Houqiang & Xu, Shenmeng & Xiao, Tingting & Hemminger, Brad M. & Yang, Siluo, 2017. "Global science discussed in local altmetrics: Weibo and its comparison with Twitter," Journal of Informetrics, Elsevier, vol. 11(2), pages 466-482.
    15. Banshal, Sumit Kumar & Gupta, Solanki & Lathabai, Hiran H & Singh, Vivek Kumar, 2022. "Power Laws in altmetrics: An empirical analysis," Journal of Informetrics, Elsevier, vol. 16(3).
    16. Bornmann, Lutz & Haunschild, Robin, 2015. "Which people use which scientific papers? An evaluation of data from F1000 and Mendeley," Journal of Informetrics, Elsevier, vol. 9(3), pages 477-487.
    17. Mingyang Wang & Zhenyu Wang & Guangsheng Chen, 2019. "Which can better predict the future success of articles? Bibliometric indices or alternative metrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(3), pages 1575-1595, June.
    18. Lutz Bornmann, 2015. "Interrater reliability and convergent validity of F1000Prime peer review," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(12), pages 2415-2426, December.
    19. Fairclough, Ruth & Thelwall, Mike, 2015. "National research impact indicators from Mendeley readers," Journal of Informetrics, Elsevier, vol. 9(4), pages 845-859.
    20. Liwei Zhang & Jue Wang, 2021. "What affects publications’ popularity on Twitter?," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(11), pages 9185-9198, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:8:y:2014:i:4:p:895-903. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.