IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v8y2014i2p421-430.html
   My bibliography  Save this article

Do Article Influence scores overestimate the citation impact of social science journals in subfields that are related to higher-impact natural science disciplines?

Author

Listed:
  • Walters, William H.

Abstract

Unlike Impact Factors (IF), Article Influence (AI) scores assign greater weight to citations that appear in highly cited journals. The natural sciences tend to have higher citation rates than the social sciences. We might therefore expect that relative to IF, AI overestimates the citation impact of social science journals in subfields that are related to (and presumably cited in) higher-impact natural science disciplines. This study evaluates that assertion through a set of simple and multiple regressions covering seven social science disciplines: anthropology, communication, economics, education, library and information science, psychology, and sociology. Contrary to expectations, AI underestimates 5IF (five-year Impact Factor) for journals in science-related subfields such as scientific communication, science education, scientometrics, biopsychology, and medical sociology. Journals in these subfields have low AI scores relative to their 5IF values. Moreover, the effect of science-related status is considerable—typically 0.60 5IF units or 0.50 SD. This effect is independent of the more general finding that AI scores underestimate 5IF for higher-impact journals. It is also independent of the very modest curvilinearity in the relationship between AI and 5IF.

Suggested Citation

  • Walters, William H., 2014. "Do Article Influence scores overestimate the citation impact of social science journals in subfields that are related to higher-impact natural science disciplines?," Journal of Informetrics, Elsevier, vol. 8(2), pages 421-430.
    • Handle: RePEc:eee:infome:v:8:y:2014:i:2:p:421-430
    DOI: 10.1016/j.joi.2014.02.001
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157714000273
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.joi.2014.02.001?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Philip M. Davis, 2008. "Eigenfactor: Does the principle of repeated improvement result in better estimates than raw citation counts?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(13), pages 2186-2188, November.
    2. Ludo Waltman & Nees Jan van Eck, 2010. "The relation between Eigenfactor, audience factor, and influence weight," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 61(7), pages 1476-1486, July.
    3. Chia-Lin Chang & Michael McAleer & Les Oxley, 2011. "What makes a great journal great in the sciences? Which came first, the chicken or the egg?," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(1), pages 17-40, April.
    4. Alexandru T. Balaban, 2012. "Positive and negative aspects of citation indices and journal impact factors," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 241-247, August.
    5. Moed, Henk F., 2010. "Measuring contextual citation impact of scientific journals," Journal of Informetrics, Elsevier, vol. 4(3), pages 265-277.
    6. Loet Leydesdorff, 2008. "Caveats for the use of citation indicators in research and journal evaluations," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(2), pages 278-287, January.
    7. Chia‐Lin Chang & Michael McAleer & Les Oxley, 2011. "What Makes A Great Journal Great In Economics? The Singer Not The Song," Journal of Economic Surveys, Wiley Blackwell, vol. 25(2), pages 326-361, April.
    8. Benjamin M. Althouse & Jevin D. West & Carl T. Bergstrom & Theodore Bergstrom, 2009. "Differences in impact factor across fields and over time," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(1), pages 27-34, January.
    9. Judit Bar-Ilan, 2012. "Journal report card," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 249-260, August.
    10. Kristie M. Engemann & Howard J. Wall, 2009. "A journal ranking for the ambitious economist," Review, Federal Reserve Bank of St. Louis, vol. 91(May), pages 127-140.
    11. Jevin West & Theodore Bergstrom & Carl T. Bergstrom, 2010. "Big Macs and Eigenfactor scores: Don't let correlation coefficients fool you," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(9), pages 1800-1807, September.
    12. Jerome K. Vanclay, 2009. "Bias in the journal impact factor," Scientometrics, Springer;Akadémiai Kiadó, vol. 78(1), pages 3-12, January.
    13. Franceschet, Massimo, 2010. "Journal influence factors," Journal of Informetrics, Elsevier, vol. 4(3), pages 239-248.
    14. Liebowitz, S J & Palmer, J P, 1984. "Assessing the Relative Impacts of Economic Journals," Journal of Economic Literature, American Economic Association, vol. 22(1), pages 77-88, March.
    15. Waltman, Ludo & van Eck, Nees Jan & van Leeuwen, Thed N. & Visser, Martijn S., 2013. "Some modifications to the SNIP journal impact indicator," Journal of Informetrics, Elsevier, vol. 7(2), pages 272-285.
    16. Mark R. Elkins & Christopher G. Maher & Robert D. Herbert & Anne M. Moseley & Catherine Sherrington, 2010. "Correlation between the Journal Impact Factor and three other journal citation indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(1), pages 81-93, October.
    17. Jonathan M. Levitt & Mike Thelwall, 2008. "Is multidisciplinary research more highly cited? A macrolevel study," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(12), pages 1973-1984, October.
    18. Thed Leeuwen, 2012. "Discussing some basic critique on Journal Impact Factors: revision of earlier comments," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 443-455, August.
    19. Ludo Waltman & Nees Jan van Eck, 2010. "The relation between Eigenfactor, audience factor, and influence weight," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(7), pages 1476-1486, July.
    20. Lawrence Smolinsky & Aaron Lercher, 2012. "Citation rates in mathematics: a study of variation by subdiscipline," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(3), pages 911-924, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    2. Dašić Predrag, 2015. "State and Analysis of Scientific Journals in the Field of “Economic Sciences” for the Period 1995-2014," Economic Themes, Sciendo, vol. 53(4), pages 547-581, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    2. Finardi, Ugo, 2013. "Correlation between Journal Impact Factor and Citation Performance: An experimental study," Journal of Informetrics, Elsevier, vol. 7(2), pages 357-370.
    3. Chang, Chia-Lin & McAleer, Michael & Oxley, Les, 2013. "Coercive journal self citations, impact factor, Journal Influence and Article Influence," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 93(C), pages 190-197.
    4. Chia-Lin Chang & Michael McAleer & Les Oxley, 2012. "Journal Impact Factor, Eigenfactor, Journal Influence and Article Influence," KIER Working Papers 822, Kyoto University, Institute of Economic Research.
    5. Liwei Cai & Jiahao Tian & Jiaying Liu & Xiaomei Bai & Ivan Lee & Xiangjie Kong & Feng Xia, 2019. "Scholarly impact assessment: a survey of citation weighting solutions," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(2), pages 453-478, February.
    6. Mingers, John & Leydesdorff, Loet, 2015. "A review of theory and practice in scientometrics," European Journal of Operational Research, Elsevier, vol. 246(1), pages 1-19.
    7. Pajić, Dejan, 2015. "On the stability of citation-based journal rankings," Journal of Informetrics, Elsevier, vol. 9(4), pages 990-1006.
    8. Lutz Bornmann & Alexander Butz & Klaus Wohlrabe, 2018. "What are the top five journals in economics? A new meta-ranking," Applied Economics, Taylor & Francis Journals, vol. 50(6), pages 659-675, February.
    9. Vanclay, Jerome K., 2012. "Publication patterns of award-winning forest scientists and implications for the Australian ERA journal ranking," Journal of Informetrics, Elsevier, vol. 6(1), pages 19-26.
    10. Waltman, Ludo & van Eck, Nees Jan, 2013. "A systematic empirical comparison of different approaches for normalizing citation impact indicators," Journal of Informetrics, Elsevier, vol. 7(4), pages 833-849.
    11. P. Dorta-González & M. I. Dorta-González, 2013. "Comparing journals from different fields of science and social science through a JCR subject categories normalized impact factor," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(2), pages 645-672, May.
    12. Chia-Lin Chang & Michael McAleer & Les Oxley, 2010. "Journal Impact Factor Versus Eigenfactor and Article Influence," KIER Working Papers 737, Kyoto University, Institute of Economic Research.
    13. David L. Anderson & John Tressler, 2017. "Researcher rank stability across alternative output measurement schemes in the context of a time limited research evaluation: the New Zealand case," Applied Economics, Taylor & Francis Journals, vol. 49(45), pages 4542-4553, September.
    14. Klaus Wohlrabe, 2016. "Taking the Temperature: A Meta-Ranking of Economics Journals," CESifo Working Paper Series 5726, CESifo.
    15. Zhou, Ping & Leydesdorff, Loet, 2011. "Fractional counting of citations in research evaluation: A cross- and interdisciplinary assessment of the Tsinghua University in Beijing," Journal of Informetrics, Elsevier, vol. 5(3), pages 360-368.
    16. Damien Besancenot & Abdelghani Maddi, 2019. "Should citations be weighted to assess the influence of an academic article?," Economics Bulletin, AccessEcon, vol. 435(1), pages 435-445.
    17. repec:hal:cepnwp:hal-01922259 is not listed on IDEAS
    18. Marcelo Mendoza, 2021. "Differences in Citation Patterns across Areas, Article Types and Age Groups of Researchers," Publications, MDPI, vol. 9(4), pages 1-23, October.
    19. Antonin Mac'e, 2017. "The Limits of Citation Counts," Papers 1711.02695, arXiv.org, revised Sep 2023.
    20. Mingers, John & Yang, Liying, 2017. "Evaluating journal quality: A review of journal citation indicators and ranking in business and management," European Journal of Operational Research, Elsevier, vol. 257(1), pages 323-337.
    21. Peter Jacso, 2012. "Grim tales about the impact factor and the h-index in the Web of Science and the Journal Citation Reports databases: reflections on Vanclay’s criticism," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 325-354, August.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:8:y:2014:i:2:p:421-430. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.