IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v108y2016i1d10.1007_s11192-016-1973-7.html
   My bibliography  Save this article

Interpreting correlations between citation counts and other indicators

Author

Listed:
  • Mike Thelwall

    () (University of Wolverhampton)

Abstract

Abstract Altmetrics or other indicators for the impact of academic outputs are often correlated with citation counts in order to help assess their value. Nevertheless, there are no guidelines about how to assess the strengths of the correlations found. This is a problem because the correlation strength affects the conclusions that should be drawn. In response, this article uses experimental simulations to assess the correlation strengths to be expected under various different conditions. The results show that the correlation strength reflects not only the underlying degree of association but also the average magnitude of the numbers involved. Overall, the results suggest that due to the number of assumptions that must be made, in practice it will rarely be possible to make a realistic interpretation of the strength of a correlation coefficient.

Suggested Citation

  • Mike Thelwall, 2016. "Interpreting correlations between citation counts and other indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(1), pages 337-347, July.
  • Handle: RePEc:spr:scient:v:108:y:2016:i:1:d:10.1007_s11192-016-1973-7
    DOI: 10.1007/s11192-016-1973-7
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-016-1973-7
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Michal Brzezinski, 2015. "Power laws in citation distributions: evidence from Scopus," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(1), pages 213-228, April.
    2. Vincent Larivière & Yves Gingras, 2010. "On the relationship between interdisciplinarity and scientific impact," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(1), pages 126-131, January.
    3. Lee Cronbach, 1951. "Coefficient alpha and the internal structure of tests," Psychometrika, Springer;The Psychometric Society, vol. 16(3), pages 297-334, September.
    4. S. Redner, 1998. "How popular is your paper? An empirical study of the citation distribution," The European Physical Journal B: Condensed Matter and Complex Systems, Springer;EDP Sciences, vol. 4(2), pages 131-134, July.
    5. Chakraborty, Tanmoy & Tammana, Vihar & Ganguly, Niloy & Mukherjee, Animesh, 2015. "Understanding and modeling diverse scientific careers of researchers," Journal of Informetrics, Elsevier, vol. 9(1), pages 69-78.
    6. Finardi, Ugo, 2013. "Correlation between Journal Impact Factor and Citation Performance: An experimental study," Journal of Informetrics, Elsevier, vol. 7(2), pages 357-370.
    7. Clément Bosquet & Pierre-Philippe Combes, 2013. "Are academics who publish more also more cited? Individual determinants of publication and citation records," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(3), pages 831-857, December.
    8. Ahlgren, Per & Waltman, Ludo, 2014. "The correlation between citation-based and expert-based assessments of publication channels: SNIP and SJR vs. Norwegian quality assessments," Journal of Informetrics, Elsevier, vol. 8(4), pages 985-996.
    9. Didegah, Fereshteh & Thelwall, Mike, 2013. "Which factors help authors produce the highest impact research? Collaboration, journal and document properties," Journal of Informetrics, Elsevier, vol. 7(4), pages 861-873.
    10. Thelwall, Mike & Fairclough, Ruth, 2015. "The influence of time and discipline on the magnitude of correlations between citation counts and quality scores," Journal of Informetrics, Elsevier, vol. 9(3), pages 529-541.
    11. Gillespie, Colin S., 2015. "Fitting Heavy Tailed Distributions: The poweRlaw Package," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 64(i02).
    12. Thelwall, Mike & Wilson, Paul, 2014. "Distributions for cited articles from individual subjects and years," Journal of Informetrics, Elsevier, vol. 8(4), pages 824-839.
    13. Natsuo Onodera & Fuyuki Yoshikane, 2015. "Factors affecting citation rates of research articles," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(4), pages 739-764, April.
    14. Thelwall, Mike, 2016. "The discretised lognormal and hooked power law distributions for complete citation data: Best options for modelling and regression," Journal of Informetrics, Elsevier, vol. 10(2), pages 336-346.
    15. Ehsan Mohammadi & Mike Thelwall, 2014. "Mendeley readership altmetrics for the social sciences and humanities: Research evaluation and knowledge flows," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(8), pages 1627-1638, August.
    16. Franceschet, Massimo & Costantini, Antonio, 2011. "The first Italian research assessment exercise: A bibliometric perspective," Journal of Informetrics, Elsevier, vol. 5(2), pages 275-291.
    17. Ajiferuke, Isola & Famoye, Felix, 2015. "Modelling count response variables in informetric studies: Comparison among count, linear, and lognormal regression models," Journal of Informetrics, Elsevier, vol. 9(3), pages 499-513.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Thelwall, Mike, 2018. "Do females create higher impact research? Scopus citations and Mendeley readers for articles from five countries," Journal of Informetrics, Elsevier, vol. 12(4), pages 1031-1041.
    2. Dunaiski, Marcel & Geldenhuys, Jaco & Visser, Willem, 2018. "How to evaluate rankings of academic entities using test data," Journal of Informetrics, Elsevier, vol. 12(3), pages 631-655.
    3. Martín-Martín, Alberto & Orduna-Malea, Enrique & Delgado López-Cózar, Emilio, 2018. "Author-level metrics in the new academic profile platforms: The online behaviour of the Bibliometrics community," Journal of Informetrics, Elsevier, vol. 12(2), pages 494-509.
    4. Ronald Snijder, 2016. "Revisiting an open access monograph experiment: measuring citations and tweets 5 years later," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1855-1875, December.
    5. Thelwall, Mike, 2018. "Microsoft Academic automatic document searches: Accuracy for journal articles and suitability for citation analysis," Journal of Informetrics, Elsevier, vol. 12(1), pages 1-9.
    6. Kim Holmberg & Julia Vainio, 2018. "Why do some research articles receive more online attention and higher altmetrics? Reasons for online success according to the authors," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(1), pages 435-447, July.
    7. Mike Thelwall, 2018. "Differences between journals and years in the proportions of students, researchers and faculty registering Mendeley articles," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(2), pages 717-729, May.
    8. Ortega, José Luis, 2018. "The life cycle of altmetric impact: A longitudinal study of six metrics from PlumX," Journal of Informetrics, Elsevier, vol. 12(3), pages 579-589.
    9. Thelwall, Mike & Nevill, Tamara, 2018. "Could scientists use Altmetric.com scores to predict longer term citation counts?," Journal of Informetrics, Elsevier, vol. 12(1), pages 237-248.
    10. Kousha, Kayvan & Thelwall, Mike & Abdoli, Mahshid, 2018. "Can Microsoft Academic assess the early citation impact of in-press articles? A multi-discipline exploratory analysis," Journal of Informetrics, Elsevier, vol. 12(1), pages 287-298.
    11. Feiheng Luo & Aixin Sun & Mojisola Erdt & Aravind Sesagiri Raamkumar & Yin-Leng Theng, 2018. "Exploring prestigious citations sourced from top universities in bibliometrics and altmetrics: a case study in the computer science discipline," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(1), pages 1-17, January.
    12. Dunaiski, Marcel & Geldenhuys, Jaco & Visser, Willem, 2018. "Author ranking evaluation at scale," Journal of Informetrics, Elsevier, vol. 12(3), pages 679-702.
    13. Mike Thelwall, 2018. "Early Mendeley readers correlate with later citation counts," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(3), pages 1231-1240, June.
    14. Kousha, Kayvan & Thelwall, Mike, 2018. "Can Microsoft Academic help to assess the citation impact of academic books?," Journal of Informetrics, Elsevier, vol. 12(3), pages 972-984.
    15. Mike Thelwall, 2018. "Does Microsoft Academic find early citations?," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(1), pages 325-334, January.
    16. Mike Thelwall, 2017. "Are Mendeley reader counts useful impact indicators in all fields?," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(3), pages 1721-1731, December.
    17. Lepori, Benedetto & Thelwall, Michael & Hoorani, Bareerah Hafeez, 2018. "Which US and European Higher Education Institutions are visible in ResearchGate and what affects their RG score?," Journal of Informetrics, Elsevier, vol. 12(3), pages 806-818.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:108:y:2016:i:1:d:10.1007_s11192-016-1973-7. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Sonal Shukla) or (Springer Nature Abstracting and Indexing). General contact details of provider: http://www.springer.com .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.