IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v9y2015i4p845-859.html
   My bibliography  Save this article

National research impact indicators from Mendeley readers

Author

Listed:
  • Fairclough, Ruth
  • Thelwall, Mike

Abstract

National research impact indicators derived from citation counts are used by governments to help assess their national research performance and to identify the effect of funding or policy changes. Citation counts lag research by several years, however, and so their information is somewhat out of date. Some of this lag can be avoided by using readership counts from the social reference sharing site Mendeley because these accumulate more quickly than citations. This article introduces a method to calculate national research impact indicators from Mendeley, using citation counts from older time periods to partially compensate for international biases in Mendeley readership. A refinement to accommodate recent national changes in Mendeley uptake makes little difference, despite being theoretically more accurate. The Mendeley patterns using the methods broadly reflect the results from similar calculations with citations and seem to reflect impact trends about a year earlier. Nevertheless, the reasons for the differences between the indicators from the two data sources are unclear.

Suggested Citation

  • Fairclough, Ruth & Thelwall, Mike, 2015. "National research impact indicators from Mendeley readers," Journal of Informetrics, Elsevier, vol. 9(4), pages 845-859.
  • Handle: RePEc:eee:infome:v:9:y:2015:i:4:p:845-859
    DOI: 10.1016/j.joi.2015.08.003
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157715300596
    Download Restriction: Full text for ScienceDirect subscribers only

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Loet Leydesdorff & Filippo Radicchi & Lutz Bornmann & Claudio Castellano & Wouter Nooy, 2013. "Field-normalized impact factors (IFs): A comparison of rescaling and fractionally counted IFs," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(11), pages 2299-2309, November.
    2. Li, Jiang & Qiao, Lili & Li, Wenyuze & Jin, Yidan, 2014. "Chinese-language articles are not biased in citations: Evidences from Chinese-English bilingual journals in Scopus and Web of Science," Journal of Informetrics, Elsevier, vol. 8(4), pages 912-916.
    3. Mu-Hsuan Huang & Chi-Shiou Lin & Dar-Zen Chen, 2011. "Counting methods, country rank changes, and counting inflation in the assessment of national research productivity and impact," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(12), pages 2427-2436, December.
    4. Manuel Trajtenberg, 1990. "A Penny for Your Quotes: Patent Citations and the Value of Innovations," RAND Journal of Economics, The RAND Corporation, vol. 21(1), pages 172-187, Spring.
    5. Maxim Engers & Joshua S. Gans & Simon Grant & Stephen King, 1999. "First-Author Conditions," Journal of Political Economy, University of Chicago Press, vol. 107(4), pages 859-883, August.
    6. Mike Thelwall & Nabeil Maflahi, 2015. "Are scholarly articles disproportionately read in their own country? An analysis of mendeley readers," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(6), pages 1124-1135, June.
    7. Michael Roach & Wesley M. Cohen, 2013. "Lens or Prism? Patent Citations as a Measure of Knowledge Flows from Public Research," Management Science, INFORMS, vol. 59(2), pages 504-525, October.
    8. Levitt, Jonathan M. & Thelwall, Mike, 2013. "Alphabetization and the skewing of first authorship towards last names early in the alphabet," Journal of Informetrics, Elsevier, vol. 7(3), pages 575-582.
    9. Waltman, Ludo & van Eck, Nees Jan, 2015. "Field-normalized citation impact indicators and the choice of an appropriate counting method," Journal of Informetrics, Elsevier, vol. 9(4), pages 872-894.
    10. Aghaei Chadegani, Arezoo & Salehi, Hadi & Md Yunus, Melor & Farhadi, Hadi & Fooladi, Masood & Farhadi, Maryam & Ale Ebrahim, Nader, 2013. "A Comparison between Two Main Academic Literature Collections: Web of Science and Scopus Databases," MPRA Paper 46898, University Library of Munich, Germany, revised 18 Mar 2013.
    11. Ehsan Mohammadi & Mike Thelwall, 2014. "Mendeley readership altmetrics for the social sciences and humanities: Research evaluation and knowledge flows," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(8), pages 1627-1638, August.
    12. Hadas Shema & Judit Bar-Ilan & Mike Thelwall, 2014. "Do blog citations correlate with a higher number of future citations? Research blogs as a potential source for alternative metrics," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(5), pages 1018-1027, May.
    13. Aksnes, Dag W. & Schneider, Jesper W. & Gunnarsson, Magnus, 2012. "Ranking national research systems by citation indicators. A comparative analysis using whole and fractionalised counting methods," Journal of Informetrics, Elsevier, vol. 6(1), pages 36-43.
    14. Ehsan Mohammadi & Mike Thelwall & Stefanie Haustein & Vincent Larivière, 2015. "Who reads research articles? An altmetrics analysis of Mendeley user categories," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(9), pages 1832-1846, September.
    15. Ludo Waltman & Clara Calero-Medina & Joost Kosten & Ed C.M. Noyons & Robert J.W. Tijssen & Nees Jan Eck & Thed N. Leeuwen & Anthony F.J. Raan & Martijn S. Visser & Paul Wouters, 2012. "The Leiden ranking 2011/2012: Data collection, indicators, and interpretation," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(12), pages 2419-2432, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Robin Haunschild & Lutz Bornmann, 2018. "Field- and time-normalization of data with many zeros: an empirical analysis using citation and Twitter data," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(2), pages 997-1012, August.
    2. Jianhua Hou & Jiantao Ye, 2020. "Are uncited papers necessarily all nonimpact papers? A quantitative analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(2), pages 1631-1662, August.
    3. Bornmann, Lutz & Haunschild, Robin, 2016. "Normalization of Mendeley reader impact on the reader- and paper-side: A comparison of the mean discipline normalized reader score (MDNRS) with the mean normalized reader score (MNRS) and bare reader ," Journal of Informetrics, Elsevier, vol. 10(3), pages 776-788.
    4. Thelwall, Mike, 2016. "The discretised lognormal and hooked power law distributions for complete citation data: Best options for modelling and regression," Journal of Informetrics, Elsevier, vol. 10(2), pages 336-346.
    5. Fairclough, Ruth & Thelwall, Mike, 2015. "More precise methods for national research citation impact comparisons," Journal of Informetrics, Elsevier, vol. 9(4), pages 895-906.
    6. Metwaly Ali Mohamed Eldakar, 2019. "Who reads international Egyptian academic articles? An altmetrics analysis of Mendeley readership categories," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(1), pages 105-135, October.
    7. Brett Buttliere & Jürgen Buder, 2017. "Personalizing papers using Altmetrics: comparing paper ‘Quality’ or ‘Impact’ to person ‘Intelligence’ or ‘Personality’," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(1), pages 219-239, April.
    8. Christopher Carroll, 2016. "Measuring academic research impact: creating a citation profile using the conceptual framework for implementation fidelity as a case study," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1329-1340, November.
    9. Haunschild, Robin & Bornmann, Lutz, 2016. "Normalization of Mendeley reader counts for impact assessment," Journal of Informetrics, Elsevier, vol. 10(1), pages 62-73.
    10. Thelwall, Mike, 2016. "The precision of the arithmetic mean, geometric mean and percentiles for citation data: An experimental simulation modelling approach," Journal of Informetrics, Elsevier, vol. 10(1), pages 110-123.
    11. Mike Thelwall, 2018. "Differences between journals and years in the proportions of students, researchers and faculty registering Mendeley articles," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(2), pages 717-729, May.
    12. A. Basu & P. Foland & G. Holdridge & R. D. Shelton, 2018. "China’s rising leadership in science and technology: quantitative and qualitative indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(1), pages 249-269, October.
    13. Zhao, Star X. & Tan, Alice M. & Yu, Shuang & Xu, Xin, 2018. "Analyzing the research funding in physics: The perspective of production and collaboration at institution level," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 508(C), pages 662-674.
    14. Jianhua Hou & Jiantao Ye, 0. "Are uncited papers necessarily all nonimpact papers? A quantitative analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 0, pages 1-32.
    15. Bornmann, Lutz & Haunschild, Robin, 2018. "Normalization of zero-inflated data: An empirical analysis of a new indicator family and its use with altmetrics data," Journal of Informetrics, Elsevier, vol. 12(3), pages 998-1011.
    16. Thelwall, Mike, 2017. "Three practical field normalised alternative indicator formulae for research evaluation," Journal of Informetrics, Elsevier, vol. 11(1), pages 128-151.
    17. Thelwall, Mike & Fairclough, Ruth, 2017. "The accuracy of confidence intervals for field normalised indicators," Journal of Informetrics, Elsevier, vol. 11(2), pages 530-540.
    18. Mojisola Erdt & Aarthy Nagarajan & Sei-Ching Joanna Sin & Yin-Leng Theng, 2016. "Altmetrics: an analysis of the state-of-the-art in measuring research impact on social media," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1117-1166, November.
    19. Siviwe Bangani, 2018. "The impact of electronic theses and dissertations: a study of the institutional repository of a university in South Africa," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 131-151, April.
    20. Radhamany Sooryamoorthy, 2019. "Scientific knowledge in South Africa: information trends, patterns and collaboration," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(3), pages 1365-1386, June.
    21. Amalia Mas-Bleda & Mike Thelwall, 2016. "Can alternative indicators overcome language biases in citation counts? A comparison of Spanish and UK research," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 2007-2030, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:9:y:2015:i:4:p:845-859. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Haili He). General contact details of provider: http://www.elsevier.com/locate/joi .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.