IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v113y2017i3d10.1007_s11192-017-2557-x.html
   My bibliography  Save this article

Are Mendeley reader counts useful impact indicators in all fields?

Author

Listed:
  • Mike Thelwall

    (University of Wolverhampton)

Abstract

Reader counts from the social reference sharing site Mendeley are known to be valuable for early research evaluation. They have strong correlations with citation counts for journal articles but appear about a year before them. There are disciplinary differences in the value of Mendeley reader counts but systematic evidence is needed at the level of narrow fields to reveal its extent. In response, this article compares Mendeley reader counts with Scopus citation counts for journal articles from 2012 in 325 narrow Scopus fields. Despite strong positive correlations in most fields, averaging 0.671, the correlations in some fields are as weak as 0.255. Technical reasons explain most weaker correlations, suggesting that the underlying relationship is almost always strong. The exceptions are caused by unusually high educational or professional use or topics of interest within countries that avoid Mendeley. The findings suggest that if care is taken then Mendeley reader counts can be used for early citation impact evidence in almost all fields and for related impact in some of the remainder. As an additional application of the results, cross-checking with Mendeley data can be used to identify indexing anomalies in citation databases.

Suggested Citation

  • Mike Thelwall, 2017. "Are Mendeley reader counts useful impact indicators in all fields?," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(3), pages 1721-1731, December.
  • Handle: RePEc:spr:scient:v:113:y:2017:i:3:d:10.1007_s11192-017-2557-x
    DOI: 10.1007/s11192-017-2557-x
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-017-2557-x
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s11192-017-2557-x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Wang, Qi & Waltman, Ludo, 2016. "Large-scale analysis of the accuracy of the journal classification systems of Web of Science and Scopus," Journal of Informetrics, Elsevier, vol. 10(2), pages 347-364.
    2. Michel Zitt, 2012. "The journal impact factor: angel, devil, or scapegoat? A comment on J.K. Vanclay’s article 2011," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 485-503, August.
    3. Fiorenzo Franceschini & Domenico Maisano & Luca Mastrogiacomo, 2015. "Errors in DOI indexing by bibliometric databases," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(3), pages 2181-2186, March.
    4. Mike Thelwall & Nabeil Maflahi, 2015. "Are scholarly articles disproportionately read in their own country? An analysis of mendeley readers," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(6), pages 1124-1135, June.
    5. Mike Thelwall & Pardeep Sud, 2016. "Mendeley readership counts: An investigation of temporal and disciplinary differences," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 67(12), pages 3036-3050, December.
    6. Halevi, Gali & Moed, Henk & Bar-Ilan, Judit, 2017. "Suitability of Google Scholar as a source of scientific information and as a source of data for scientific evaluation—Review of the Literature," Journal of Informetrics, Elsevier, vol. 11(3), pages 823-834.
    7. Philippe Mongeon & Adèle Paul-Hus, 2016. "The journal coverage of Web of Science and Scopus: a comparative analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(1), pages 213-228, January.
    8. Mike Thelwall & Paul Wilson, 2016. "Mendeley readership altmetrics for medical articles: An analysis of 45 fields," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 67(8), pages 1962-1972, August.
    9. Pardeep Sud & Mike Thelwall, 2014. "Evaluating altmetrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(2), pages 1131-1143, February.
    10. Zohreh Zahedi & Rodrigo Costas & Paul Wouters, 2014. "How well developed are altmetrics? A cross-disciplinary analysis of the presence of ‘alternative metrics’ in scientific publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1491-1513, November.
    11. Anne-Wil Harzing & Satu Alakangas, 2017. "Microsoft Academic is one year old: the Phoenix is ready to leave the nest," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(3), pages 1887-1894, September.
    12. Nabeil Maflahi & Mike Thelwall, 2016. "When are readership counts as useful as citation counts? Scopus versus Mendeley for LIS journals," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 67(1), pages 191-199, January.
    13. Anthony F. J. Raan, 2006. "Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups," Scientometrics, Springer;Akadémiai Kiadó, vol. 67(3), pages 491-502, June.
    14. Thelwall, Mike, 2016. "Are there too many uncited articles? Zero inflated variants of the discretised lognormal and hooked power law distributions," Journal of Informetrics, Elsevier, vol. 10(2), pages 622-633.
    15. Zohreh Zahedi & Rodrigo Costas & Paul Wouters, 2017. "Mendeley readership as a filtering tool to identify highly cited publications," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 68(10), pages 2511-2521, October.
    16. Judit Bar-Ilan, 2014. "Astrophysics publications on arXiv, Scopus and Mendeley: a case study," Scientometrics, Springer;Akadémiai Kiadó, vol. 100(1), pages 217-225, July.
    17. Gorraiz, Juan & Melero-Fuentes, David & Gumpenberger, Christian & Valderrama-Zurián, Juan-Carlos, 2016. "Availability of digital object identifiers (DOIs) in Web of Science and Scopus," Journal of Informetrics, Elsevier, vol. 10(1), pages 98-109.
    18. Thelwall, Mike, 2017. "Three practical field normalised alternative indicator formulae for research evaluation," Journal of Informetrics, Elsevier, vol. 11(1), pages 128-151.
    19. Ehsan Mohammadi & Mike Thelwall & Kayvan Kousha, 2016. "Can Mendeley bookmarks reflect readership? A survey of user motivations," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 67(5), pages 1198-1209, May.
    20. Mike Thelwall, 2016. "Interpreting correlations between citation counts and other indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(1), pages 337-347, July.
    21. Ehsan Mohammadi & Mike Thelwall, 2014. "Mendeley readership altmetrics for the social sciences and humanities: Research evaluation and knowledge flows," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(8), pages 1627-1638, August.
    22. Ehsan Mohammadi & Mike Thelwall & Stefanie Haustein & Vincent Larivière, 2015. "Who reads research articles? An altmetrics analysis of Mendeley user categories," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(9), pages 1832-1846, September.
    23. Thelwall, Mike & Fairclough, Ruth, 2015. "Geometric journal impact factors correcting for individual highly cited articles," Journal of Informetrics, Elsevier, vol. 9(2), pages 263-272.
    24. Dag W Aksnes & Randi Elisabeth Taxt, 2004. "Peer reviews and bibliometric indicators: a comparative study at a Norwegian university," Research Evaluation, Oxford University Press, vol. 13(1), pages 33-41, April.
    25. Abramo, Giovanni & Cicero, Tindaro & D’Angelo, Ciriaco Andrea, 2011. "Assessing the varying level of impact measurement accuracy as a function of the citation window length," Journal of Informetrics, Elsevier, vol. 5(4), pages 659-667.
    26. Juan Miguel Campanario, 2011. "Empirical study of journal impact factors obtained using the classical two-year citation window versus a five-year citation window," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(1), pages 189-204, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Thelwall, Mike, 2018. "Do females create higher impact research? Scopus citations and Mendeley readers for articles from five countries," Journal of Informetrics, Elsevier, vol. 12(4), pages 1031-1041.
    2. Carlos Vílchez-Román & Arístides Vara-Horna, 2021. "Usage, content and citation in open access publication: any interaction effects?," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(12), pages 9457-9476, December.
    3. Mike Thelwall, 2020. "Data in Brief: Can a mega-journal for data be useful?," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(1), pages 697-709, July.
    4. Lutz Bornmann & Rüdiger Mutz & Robin Haunschild & Felix Moya-Anegon & Mirko Almeida Madeira Clemente & Moritz Stefaner, 2021. "Mapping the impact of papers on various status groups in excellencemapping.net: a new release of the excellence mapping tool based on citation and reader scores," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(11), pages 9305-9331, November.
    5. Zahedi, Zohreh & Haustein, Stefanie, 2018. "On the relationships between bibliographic characteristics of scientific documents and citation and Mendeley readership counts: A large-scale analysis of Web of Science publications," Journal of Informetrics, Elsevier, vol. 12(1), pages 191-202.
    6. Thelwall, Mike & Nevill, Tamara, 2018. "Could scientists use Altmetric.com scores to predict longer term citation counts?," Journal of Informetrics, Elsevier, vol. 12(1), pages 237-248.
    7. Paul Kudlow & Devin Bissky Dziadyk & Alan Rutledge & Aviv Shachak & Gunther Eysenbach, 2020. "The citation advantage of promoted articles in a cross‐publisher distribution platform: A 12‐month randomized controlled trial," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 71(10), pages 1257-1274, October.
    8. S. Ravikumar & Bidyut Bikash Boruah & M. N. Ravikumar, 2022. "Correlation study between citation count and Mendeley readership of the articles of Sri Lankan authors," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(8), pages 4873-4885, August.
    9. Maryam Moshtagh & Tahereh Jowkar & Maryam Yaghtin & Hajar Sotudeh, 2023. "The moderating effect of altmetrics on the correlations between single and multi-faceted university ranking systems: the case of THE and QS vs. Nature Index and Leiden," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(1), pages 761-781, January.
    10. Zhiqi Wang & Wolfgang Glänzel & Yue Chen, 2020. "The impact of preprints in Library and Information Science: an analysis of citations, usage and social attention indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1403-1423, November.
    11. Rafael Repiso & Antonio Castillo-Esparcia & Daniel Torres-Salinas, 2019. "Altmetrics, alternative indicators for Web of Science Communication studies journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(2), pages 941-958, May.
    12. Mike Thelwall, 2018. "Early Mendeley readers correlate with later citation counts," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(3), pages 1231-1240, June.
    13. Kim Holmberg & Han Woo Park, 2018. "An altmetric investigation of the online visibility of South Korea-based scientific journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(1), pages 603-613, October.
    14. Sergio Copiello, 2020. "Other than detecting impact in advance, alternative metrics could act as early warning signs of retractions: tentative findings of a study into the papers retracted by PLoS ONE," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2449-2469, December.
    15. Xiaozan Lyu & Rodrigo Costas, 2020. "How do academic topics shift across altmetric sources? A case study of the research area of Big Data," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(2), pages 909-943, May.
    16. Thelwall, Mike, 2018. "Dimensions: A competitor to Scopus and the Web of Science?," Journal of Informetrics, Elsevier, vol. 12(2), pages 430-435.
    17. Michael Thelwall, 2018. "Can Microsoft Academic be used for citation analysis of preprint archives? The case of the Social Science Research Network," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(2), pages 913-928, May.
    18. Zhang, Xinyuan & Xie, Qing & Song, Min, 2021. "Measuring the impact of novelty, bibliometric, and academic-network factors on citation count using a neural network," Journal of Informetrics, Elsevier, vol. 15(2).
    19. Wolfgang Glänzel & Pei-Shan Chi, 2020. "The big challenge of Scientometrics 2.0: exploring the broader impact of scientific research in public health," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1011-1031, November.
    20. Abramo, Giovanni, 2018. "Revisiting the scientometric conceptualization of impact and its measurement," Journal of Informetrics, Elsevier, vol. 12(3), pages 590-597.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mike Thelwall, 2018. "Differences between journals and years in the proportions of students, researchers and faculty registering Mendeley articles," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(2), pages 717-729, May.
    2. Mike Thelwall, 2018. "Early Mendeley readers correlate with later citation counts," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(3), pages 1231-1240, June.
    3. Thelwall, Mike, 2018. "Do females create higher impact research? Scopus citations and Mendeley readers for articles from five countries," Journal of Informetrics, Elsevier, vol. 12(4), pages 1031-1041.
    4. Thelwall, Mike, 2018. "Microsoft Academic automatic document searches: Accuracy for journal articles and suitability for citation analysis," Journal of Informetrics, Elsevier, vol. 12(1), pages 1-9.
    5. Michael Thelwall, 2018. "Can Microsoft Academic be used for citation analysis of preprint archives? The case of the Social Science Research Network," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(2), pages 913-928, May.
    6. Mike Thelwall, 2018. "Does Microsoft Academic find early citations?," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(1), pages 325-334, January.
    7. Kousha, Kayvan & Thelwall, Mike & Abdoli, Mahshid, 2018. "Can Microsoft Academic assess the early citation impact of in-press articles? A multi-discipline exploratory analysis," Journal of Informetrics, Elsevier, vol. 12(1), pages 287-298.
    8. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    9. Metwaly Ali Mohamed Eldakar, 2019. "Who reads international Egyptian academic articles? An altmetrics analysis of Mendeley readership categories," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(1), pages 105-135, October.
    10. Ortega, José Luis, 2018. "The life cycle of altmetric impact: A longitudinal study of six metrics from PlumX," Journal of Informetrics, Elsevier, vol. 12(3), pages 579-589.
    11. Kaltrina Nuredini, 2021. "Investigating Altmetric Information For The Top 1000 Journals From Handelsblatt Ranking In Economic And Business Studies," Journal of Economic Surveys, Wiley Blackwell, vol. 35(5), pages 1315-1343, December.
    12. Mojisola Erdt & Aarthy Nagarajan & Sei-Ching Joanna Sin & Yin-Leng Theng, 2016. "Altmetrics: an analysis of the state-of-the-art in measuring research impact on social media," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1117-1166, November.
    13. Mike Thelwall & Kayvan Kousha & Mahshid Abdoli & Emma Stuart & Meiko Makita & Paul Wilson & Jonathan Levitt, 2023. "Do altmetric scores reflect article quality? Evidence from the UK Research Excellence Framework 2021," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 74(5), pages 582-593, May.
    14. Raminta Pranckutė, 2021. "Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World," Publications, MDPI, vol. 9(1), pages 1-59, March.
    15. Thelwall, Mike & Nevill, Tamara, 2018. "Could scientists use Altmetric.com scores to predict longer term citation counts?," Journal of Informetrics, Elsevier, vol. 12(1), pages 237-248.
    16. Banshal, Sumit Kumar & Gupta, Solanki & Lathabai, Hiran H & Singh, Vivek Kumar, 2022. "Power Laws in altmetrics: An empirical analysis," Journal of Informetrics, Elsevier, vol. 16(3).
    17. Wang, Zhiqi & Chen, Yue & Glänzel, Wolfgang, 2020. "Preprints as accelerator of scholarly communication: An empirical analysis in Mathematics," Journal of Informetrics, Elsevier, vol. 14(4).
    18. Mike Thelwall, 2021. "Measuring Societal Impacts Of Research With Altmetrics? Common Problems And Mistakes," Journal of Economic Surveys, Wiley Blackwell, vol. 35(5), pages 1302-1314, December.
    19. Amalia Mas-Bleda & Mike Thelwall, 2016. "Can alternative indicators overcome language biases in citation counts? A comparison of Spanish and UK research," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 2007-2030, December.
    20. Maryam Moshtagh & Tahereh Jowkar & Maryam Yaghtin & Hajar Sotudeh, 2023. "The moderating effect of altmetrics on the correlations between single and multi-faceted university ranking systems: the case of THE and QS vs. Nature Index and Leiden," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(1), pages 761-781, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:113:y:2017:i:3:d:10.1007_s11192-017-2557-x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.