IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v12y2018i1p237-248.html
   My bibliography  Save this article

Could scientists use Altmetric.com scores to predict longer term citation counts?

Author

Listed:
  • Thelwall, Mike
  • Nevill, Tamara

Abstract

Altmetrics from Altmetric.com are widely used by publishers and researchers to give earlier evidence of attention than citation counts. This article assesses whether Altmetric.com scores are reliable early indicators of likely future impact and whether they may also reflect non-scholarly impacts. A preliminary factor analysis suggests that the main altmetric indicator of scholarly impact is Mendeley reader counts, with weaker news, informational and social network discussion/promotion dimensions in some fields. Based on a regression analysis of Altmetric.com data from November 2015 and Scopus citation counts from October 2017 for articles in 30 narrow fields, only Mendeley reader counts are consistent predictors of future citation impact. Most other Altmetric.com scores can help predict future impact in some fields. Overall, the results confirm that early Altmetric.com scores can predict later citation counts, although less well than journal impact factors, and the optimal strategy is to consider both Altmetric.com scores and journal impact factors. Altmetric.com scores can also reflect dimensions of non-scholarly impact in some fields.

Suggested Citation

  • Thelwall, Mike & Nevill, Tamara, 2018. "Could scientists use Altmetric.com scores to predict longer term citation counts?," Journal of Informetrics, Elsevier, vol. 12(1), pages 237-248.
  • Handle: RePEc:eee:infome:v:12:y:2018:i:1:p:237-248
    DOI: 10.1016/j.joi.2018.01.008
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157717303905
    Download Restriction: Full text for ScienceDirect subscribers only
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Thelwall, Mike, 2016. "The discretised lognormal and hooked power law distributions for complete citation data: Best options for modelling and regression," Journal of Informetrics, Elsevier, vol. 10(2), pages 336-346.
    2. Thelwall, Mike & Wilson, Paul, 2014. "Regression for citation data: An evaluation of different methods," Journal of Informetrics, Elsevier, vol. 8(4), pages 963-971.
    3. Stegehuis, Clara & Litvak, Nelly & Waltman, Ludo, 2015. "Predicting the long-term citation impact of recent publications," Journal of Informetrics, Elsevier, vol. 9(3), pages 642-657.
    4. Mike Thelwall & Paul Wilson, 2016. "Mendeley readership altmetrics for medical articles: An analysis of 45 fields," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 67(8), pages 1962-1972, August.
    5. Brett Buttliere & Jürgen Buder, 2017. "Personalizing papers using Altmetrics: comparing paper ‘Quality’ or ‘Impact’ to person ‘Intelligence’ or ‘Personality’," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(1), pages 219-239, April.
    6. Mike Thelwall, 2017. "Are Mendeley reader counts useful impact indicators in all fields?," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(3), pages 1721-1731, December.
    7. M.H. MacRoberts & B.R. MacRoberts, 2010. "Problems of citation analysis: A study of uncited and seldom-cited influences," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(1), pages 1-12, January.
    8. Pardeep Sud & Mike Thelwall, 2014. "Evaluating altmetrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(2), pages 1131-1143, February.
    9. David I Stern, 2014. "High-Ranked Social Science Journal Articles Can Be Identified from Early Citation Information," PLOS ONE, Public Library of Science, vol. 9(11), pages 1-11, November.
    10. Kayvan Kousha & Mike Thelwall, 2017. "Are wikipedia citations important evidence of the impact of scholarly articles and books?," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 68(3), pages 762-779, March.
    11. Hadas Shema & Judit Bar-Ilan & Mike Thelwall, 2015. "How is research blogged? A content analysis approach," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(6), pages 1136-1149, June.
    12. Zohreh Zahedi & Rodrigo Costas & Paul Wouters, 2014. "How well developed are altmetrics? A cross-disciplinary analysis of the presence of ‘alternative metrics’ in scientific publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1491-1513, November.
    13. Kim Holmberg & Mike Thelwall, 2014. "Disciplinary differences in Twitter scholarly communication," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1027-1042, November.
    14. Mike Thelwall, 2016. "Interpreting correlations between citation counts and other indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(1), pages 337-347, July.
    15. Jonathan Adams, 2005. "Early citation counts correlate with accumulated impact," Scientometrics, Springer;Akadémiai Kiadó, vol. 63(3), pages 567-581, June.
    16. George A. Lozano & Vincent Larivière & Yves Gingras, 2012. "The weakening relationship between the impact factor and papers' citations in the digital age," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(11), pages 2140-2145, November.
    17. Mike Thelwall & Nabeil Maflahi, 2015. "Are scholarly articles disproportionately read in their own country? An analysis of mendeley readers," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(6), pages 1124-1135, June.
    18. J Mingers, 2008. "Exploring the dynamics of journal citations: Modelling with s-curves," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 59(8), pages 1013-1025, August.
    19. Jian Wang, 2013. "Citation time window choice for research impact evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(3), pages 851-872, March.
    20. Franceschet, Massimo & Costantini, Antonio, 2011. "The first Italian research assessment exercise: A bibliometric perspective," Journal of Informetrics, Elsevier, vol. 5(2), pages 275-291.
    21. Rodrigo Costas & Zohreh Zahedi & Paul Wouters, 2015. "Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(10), pages 2003-2019, October.
    22. Kuang-hua Chen & Muh-chyun Tang & Chun-mei Wang & Jieh Hsiang, 2015. "Exploring alternative metrics of scholarly performance in the social sciences and humanities in Taiwan," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 97-112, January.
    23. Xianwen Wang & Wenli Mao & Shenmeng Xu & Chunbo Zhang, 2014. "Usage history of scientific literature: Nature metrics and metrics of Nature publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(3), pages 1923-1933, March.
    24. Stephan B. Bruns & David I. Stern, 2016. "Research assessment using early citation information," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(2), pages 917-935, August.
    25. Ehsan Mohammadi & Mike Thelwall & Kayvan Kousha, 2016. "Can Mendeley bookmarks reflect readership? A survey of user motivations," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 67(5), pages 1198-1209, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Cristina López-Duarte & Marta M. Vidal-Suárez & Belén González-Díaz, 2019. "Cross-national distance and international business: an analysis of the most influential recent models," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(1), pages 173-208, October.
    2. Paul Kudlow & Devin Bissky Dziadyk & Alan Rutledge & Aviv Shachak & Gunther Eysenbach, 2020. "The citation advantage of promoted articles in a cross‐publisher distribution platform: A 12‐month randomized controlled trial," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 71(10), pages 1257-1274, October.
    3. Rongying Zhao & Xu Wang, 2019. "Evaluation and comparison of influence in international Open Access journals between China and USA," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(3), pages 1091-1110, September.
    4. Anwar Said & Timothy D. Bowman & Rabeeh Ayaz Abbasi & Naif Radi Aljohani & Saeed-Ul Hassan & Raheel Nawaz, 2019. "Mining network-level properties of Twitter altmetrics data," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(1), pages 217-235, July.
    5. Kong, Ling & Wang, Dongbo, 2020. "Comparison of citations and attention of cover and non-cover papers," Journal of Informetrics, Elsevier, vol. 14(4).
    6. Sergio Copiello, 2020. "Other than detecting impact in advance, alternative metrics could act as early warning signs of retractions: tentative findings of a study into the papers retracted by PLoS ONE," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2449-2469, December.
    7. Jianhua Hou & Da Ma, 2020. "How the high-impact papers formed? A study using data from social media and citation," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2597-2615, December.
    8. Mousumi Karmakar & Sumit Kumar Banshal & Vivek Kumar Singh, 0. "Does presence of social media plugins in a journal website result in higher social media attention of its research publications?," Scientometrics, Springer;Akadémiai Kiadó, vol. 0, pages 1-41.
    9. Wang, Xing & Zhang, Zhihui, 2020. "Improving the reliability of short-term citation impact indicators by taking into account the correlation between short- and long-term citation impact," Journal of Informetrics, Elsevier, vol. 14(2).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mike Thelwall, 2018. "Early Mendeley readers correlate with later citation counts," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(3), pages 1231-1240, June.
    2. Abramo, Giovanni, 2018. "Revisiting the scientometric conceptualization of impact and its measurement," Journal of Informetrics, Elsevier, vol. 12(3), pages 590-597.
    3. Zahedi, Zohreh & Haustein, Stefanie, 2018. "On the relationships between bibliographic characteristics of scientific documents and citation and Mendeley readership counts: A large-scale analysis of Web of Science publications," Journal of Informetrics, Elsevier, vol. 12(1), pages 191-202.
    4. Zhiqi Wang & Wolfgang Glänzel & Yue Chen, 0. "The impact of preprints in Library and Information Science: an analysis of citations, usage and social attention indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 0, pages 1-21.
    5. Abramo, Giovanni & D’Angelo, Ciriaco Andrea & Felici, Giovanni, 2019. "Predicting publication long-term impact through a combination of early citations and journal impact factor," Journal of Informetrics, Elsevier, vol. 13(1), pages 32-49.
    6. Mike Thelwall, 2017. "Are Mendeley reader counts useful impact indicators in all fields?," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(3), pages 1721-1731, December.
    7. Mike Thelwall, 2018. "Differences between journals and years in the proportions of students, researchers and faculty registering Mendeley articles," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(2), pages 717-729, May.
    8. Thelwall, Mike, 2018. "Do females create higher impact research? Scopus citations and Mendeley readers for articles from five countries," Journal of Informetrics, Elsevier, vol. 12(4), pages 1031-1041.
    9. Kousha, Kayvan & Thelwall, Mike & Abdoli, Mahshid, 2018. "Can Microsoft Academic assess the early citation impact of in-press articles? A multi-discipline exploratory analysis," Journal of Informetrics, Elsevier, vol. 12(1), pages 287-298.
    10. Zhiqi Wang & Wolfgang Glänzel & Yue Chen, 2020. "The impact of preprints in Library and Information Science: an analysis of citations, usage and social attention indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1403-1423, November.
    11. Michael Thelwall, 2018. "Can Microsoft Academic be used for citation analysis of preprint archives? The case of the Social Science Research Network," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(2), pages 913-928, May.
    12. Vasilios D. Kosteas, 2018. "Predicting long-run citation counts for articles in top economics journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(3), pages 1395-1412, June.
    13. Xi Zhang & Xianhai Wang & Hongke Zhao & Patricia Ordóñez de Pablos & Yongqiang Sun & Hui Xiong, 2019. "An effectiveness analysis of altmetrics indices for different levels of artificial intelligence publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(3), pages 1311-1344, June.
    14. Sergio Copiello, 2020. "Other than detecting impact in advance, alternative metrics could act as early warning signs of retractions: tentative findings of a study into the papers retracted by PLoS ONE," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2449-2469, December.
    15. David L. Anderson & John Tressler, 2016. "Citation-Capture Rates for Economics Journals: Do they Differ from Other Disciplines and Does it Matter?," Economic Papers, The Economic Society of Australia, vol. 35(1), pages 73-85, March.
    16. Amalia Mas-Bleda & Mike Thelwall, 2016. "Can alternative indicators overcome language biases in citation counts? A comparison of Spanish and UK research," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 2007-2030, December.
    17. Thelwall, Mike, 2017. "Three practical field normalised alternative indicator formulae for research evaluation," Journal of Informetrics, Elsevier, vol. 11(1), pages 128-151.
    18. Saeed-Ul Hassan & Mubashir Imran & Uzair Gillani & Naif Radi Aljohani & Timothy D. Bowman & Fereshteh Didegah, 2017. "Measuring social media activity of scientific literature: an exhaustive comparison of scopus and novel altmetrics big data," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 1037-1057, November.
    19. Mojisola Erdt & Aarthy Nagarajan & Sei-Ching Joanna Sin & Yin-Leng Theng, 2016. "Altmetrics: an analysis of the state-of-the-art in measuring research impact on social media," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1117-1166, November.
    20. de Carvalho, Gustavo Dambiski Gomes & Sokulski, Carla Cristiane & da Silva, Wesley Vieira & de Carvalho, Hélio Gomes & de Moura, Rafael Vignoli & de Francisco, Antonio Carlos & da Veiga, Claudimar Per, 2020. "Bibliometrics and systematic reviews: A comparison between the Proknow-C and the Methodi Ordinatio," Journal of Informetrics, Elsevier, vol. 14(3).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:12:y:2018:i:1:p:237-248. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Nithya Sathishkumar). General contact details of provider: http://www.elsevier.com/locate/joi .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.