IDEAS home Printed from https://ideas.repec.org/a/bla/jamist/v61y2010i9p1800-1807.html
   My bibliography  Save this article

Big Macs and Eigenfactor scores: Don't let correlation coefficients fool you

Author

Listed:
  • Jevin West
  • Theodore Bergstrom
  • Carl T. Bergstrom

Abstract

The Eigenfactor™ Metrics provide an alternative way of evaluating scholarly journals based on an iterative ranking procedure analogous to Google's PageRank algorithm. These metrics have recently been adopted by Thomson Reuters and are listed alongside the Impact Factor in the Journal Citation Reports. But do these metrics differ sufficiently so as to be a useful addition to the bibliometric toolbox? Davis (2008) has argued that they do not, based on his finding of a 0.95 correlation coefficient between Eigenfactor score and Total Citations for a sample of journals in the field of medicine. This conclusion is mistaken; in this article, we illustrate the basic statistical fallacy to which Davis succumbed. We provide a complete analysis of the 2006 Journal Citation Reports and demonstrate that there are statistically and economically significant differences between the information provided by the Eigenfactor Metrics and that provided by Impact Factor and Total Citations.

Suggested Citation

  • Jevin West & Theodore Bergstrom & Carl T. Bergstrom, 2010. "Big Macs and Eigenfactor scores: Don't let correlation coefficients fool you," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 61(9), pages 1800-1807, September.
    • Handle: RePEc:bla:jamist:v:61:y:2010:i:9:p:1800-1807
    DOI: 10.1002/asi.21374
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/asi.21374
    Download Restriction: no
    File URL: https://libkey.io/10.1002/asi.21374?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Theodore Eisenberg & Martin T. Wells, 2014. "Ranking Law Journals And The Limits Of Journal Citation Reports," Economic Inquiry, Western Economic Association International, vol. 52(4), pages 1301-1314, October.
    2. Mingers, John & Leydesdorff, Loet, 2015. "A review of theory and practice in scientometrics," European Journal of Operational Research, Elsevier, vol. 246(1), pages 1-19.
    3. Petridis, Konstantinos & Malesios, Chrisovalantis & Arabatzis, Garyfallos & Thanassoulis, Emmanuel, 2013. "Efficiency analysis of forestry journals: Suggestions for improving journals’ quality," Journal of Informetrics, Elsevier, vol. 7(2), pages 505-521.
    4. Geoffrey S. Shideler & Rafael J. Araújo, 2016. "Measures of scholarly journal quality are not universally applicable to determining value of advertised annual subscription price," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(3), pages 963-973, June.
    5. Massucci, Francesco Alessandro & Docampo, Domingo, 2019. "Measuring the academic reputation through citation networks via PageRank," Journal of Informetrics, Elsevier, vol. 13(1), pages 185-201.
    6. Nataliya N. Matveeva & Oleg V. Poldin, 2017. "How Network Characteristics of Researchers Relate to Their Citation Indicators – a Co-Authorship Network Analysis Based on Google Scholar," HSE Working papers WP BRP 44/EDU/2017, National Research University Higher School of Economics.
    7. V. A. Traag & L. Waltman, 2019. "Systematic analysis of agreement between metrics and peer review in the UK REF," Palgrave Communications, Palgrave Macmillan, vol. 5(1), pages 1-12, December.
    8. Matveeva, Nataliya & Poldin, Oleg, 2016. "Citation of scholars in co-authorship network: Analysis of Google Scholar data," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 44, pages 100-118.
    9. Oğuzhan Alaşehir & Murat Perit Çakır & Cengiz Acartürk & Nazife Baykal & Ural Akbulut, 2014. "URAP-TR: a national ranking for Turkish universities based on academic performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 159-178, October.
    10. Filippo Radicchi, 2011. "Who Is the Best Player Ever? A Complex Network Analysis of the History of Professional Tennis," PLOS ONE, Public Library of Science, vol. 6(2), pages 1-7, February.
    11. Yu Zhang & Min Wang & Morteza Saberi & Elizabeth Chang, 2022. "Analysing academic paper ranking algorithms using test data and benchmarks: an investigation," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(7), pages 4045-4074, July.
    12. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    13. Mingers, John & Yang, Liying, 2017. "Evaluating journal quality: A review of journal citation indicators and ranking in business and management," European Journal of Operational Research, Elsevier, vol. 257(1), pages 323-337.
    14. Walters, William H., 2014. "Do Article Influence scores overestimate the citation impact of social science journals in subfields that are related to higher-impact natural science disciplines?," Journal of Informetrics, Elsevier, vol. 8(2), pages 421-430.
    15. Filippo Radicchi & Claudio Castellano, 2013. "Analysis of bibliometric indicators for individual scholars in a large data set," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(3), pages 627-637, December.
    16. Wood-Doughty, Alex & Bergstrom, Ted & Steigerwald, Douglas, 2017. "Do download reports reliably measure journal usage? Trusting the fox to count your Hens?," University of California at Santa Barbara, Economics Working Paper Series qt1f221007, Department of Economics, UC Santa Barbara.
    17. Haustein, Stefanie & Siebenlist, Tobias, 2011. "Applying social bookmarking data to evaluate journal usage," Journal of Informetrics, Elsevier, vol. 5(3), pages 446-457.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jamist:v:61:y:2010:i:9:p:1800-1807. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.asis.org .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.