IDEAS home Printed from https://ideas.repec.org/a/bpj/bejeap/v9y2009i1n34.html
   My bibliography  Save this article

Don't Get Skewed Over by Journal Rankings

Author

Listed:
  • Wall Howard J

    () (Federal Reserve Bank of St. Louis)

Abstract

Nearly all journal rankings in economics use some weighted average of citations to calculate a journal's impact. These rankings are often used, formally or informally, to help assess the publication success of individual economists or institutions. Although ranking methods and opinions are legion, scant attention has been paid to the usefulness of any ranking as representative of the many articles published in a journal. First, because the distributions of citations across articles within a journal are seriously skewed, and the skewness differs across journals, the appropriate measure of central tendency is the median rather than the mean. Second, large shares of articles in the highest-ranked journals are cited less frequently than typical articles in much-lower-ranked journals.

Suggested Citation

  • Wall Howard J, 2009. "Don't Get Skewed Over by Journal Rankings," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 9(1), pages 1-12, August.
    • Handle: RePEc:bpj:bejeap:v:9:y:2009:i:1:n:34
    as

    Download full text from publisher

    File URL: https://www.degruyter.com/view/j/bejeap.2009.9.1/bejeap.2009.9.1.2280/bejeap.2009.9.1.2280.xml?format=INT
    Download Restriction: For access to full text, subscription to the journal or payment for the individual article is required.

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Sandra Rousseau & Tom Verbeke & Ronald Rousseau, 2009. "Evaluating Environmental and Resource Economics Journals: A TOP-Curve Approach," Review of Environmental Economics and Policy, Association of Environmental and Resource Economists, vol. 3(2), pages 270-287, Summer.
    2. Andrew J. Oswald, 2007. "An Examination of the Reliability of Prestigious Scholarly Journals: Evidence and Implications for Decision‐Makers," Economica, London School of Economics and Political Science, vol. 74(293), pages 21-31, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    RePEc Biblio mentions

    As found on the RePEc Biblio, the curated bibliography for Economics:
    1. > Economics Profession > Ranking in Economics > Ranking Journals

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Lutz Bornmann & Alexander Butz & Klaus Wohlrabe, 2018. "What are the top five journals in economics? A new meta-ranking," Applied Economics, Taylor & Francis Journals, vol. 50(6), pages 659-675, February.
    2. Lando, Tommaso & Bertoli-Barsotti, Lucio, 2017. "Measuring the citation impact of journals with generalized Lorenz curves," Journal of Informetrics, Elsevier, vol. 11(3), pages 689-703.
    3. David L. Anderson & John Tressler, 2010. "The Merits of Using Citation-Based Journal Weighting Schemes to Measure Research Performance in Economics: The Case of New Zealand," Working Papers in Economics 10/03, University of Waikato.
    4. Christian Seiler & Klaus Wohlrabe, 2014. "How Robust are Journal Rankings Based on the Impact Factor? Evidence from the Economic Sciences," CESifo Working Paper Series 5036, CESifo.
    5. Justus Haucap & Johannes Muck, 2015. "What drives the relevance and reputation of economics journals? An update from a survey among economists," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(3), pages 849-877, June.
    6. Wohlrabe, Klaus, 2016. "Taking the Temperature: A Meta-Ranking of Economics Journals," MPRA Paper 68933, University Library of Munich, Germany.
    7. John Tressler & David L. Anderson, 2012. "Citations as a Measure of the Research Outputs of New Zealand's Economics Departments: The Problem of 'Long and Variable Lags'," Agenda - A Journal of Policy Analysis and Reform, Australian National University, College of Business and Economics, School of Economics, vol. 19(1), pages 17-40.
    8. Klaus Wohlrabe, 2016. "Taking the Temperature: A Meta-Ranking of Economics Journals," CESifo Working Paper Series 5726, CESifo.
    9. Bräuninger, Michael & Haucap, Justus & Muck, Johannes, 2011. "Was lesen und schätzen Ökonomen im Jahr 2011?," DICE Ordnungspolitische Perspektiven 18, University of Düsseldorf, Düsseldorf Institute for Competition Economics (DICE).
    10. Antonoyiannakis, Manolis, 2018. "Impact Factors and the Central Limit Theorem: Why citation averages are scale dependent," Journal of Informetrics, Elsevier, vol. 12(4), pages 1072-1088.
    11. David L. Anderson & John Tressler, 2011. "The Merits of Using Citations to Measure Research Output in Economics Departments: The New Zealand Case," Working Papers in Economics 11/11, University of Waikato.
    12. Seiler, Christian & Wohlrabe, Klaus, 2014. "How robust are journal rankings based on the impact factor? Evidence from the economic sciences," Journal of Informetrics, Elsevier, vol. 8(4), pages 904-911.
    13. Hofmeister Robert & Krapf Matthias, 2011. "How Do Editors Select Papers, and How Good are They at Doing It?," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 11(1), pages 1-23, October.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bpj:bejeap:v:9:y:2009:i:1:n:34. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Peter Golla). General contact details of provider: https://www.degruyter.com .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.