IDEAS home Printed from https://ideas.repec.org/p/fip/fedlwp/2009-014.html
   My bibliography  Save this paper

Journal rankings in economics: handle with care

Author

Listed:
  • Howard J. Wall

Abstract

Nearly all journal rankings in economics use some weighted average of citations to calculate a journal's impact. These rankings are often used, formally or informally, to help assess the publication success of individual economists or institutions. Although ranking methods and opinions are legion, scant attention has been paid to the usefulness of any ranking as representative of the many articles published in a journal. First, because the distributions of citations across articles within a journal are seriously skewed, and the skewness differs across journals, the appropriate measure of central tendency is the median rather than the mean. Second, large shares of articles in the highest-ranked journals are cited less frequently than typical articles in much-lower-ranked journals. Finally, a ranking that uses the h-index is very similar to one that uses total citations, making it less than ideal for assessing the typical impact of articles within a journal.

Suggested Citation

  • Howard J. Wall, 2009. "Journal rankings in economics: handle with care," Working Papers 2009-014, Federal Reserve Bank of St. Louis.
    • Handle: RePEc:fip:fedlwp:2009-014
    as

    Download full text from publisher

    File URL: http://research.stlouisfed.org/wp/2009/2009-014.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Andrew J. Oswald, 2007. "An Examination of the Reliability of Prestigious Scholarly Journals: Evidence and Implications for Decision‐Makers," Economica, London School of Economics and Political Science, vol. 74(293), pages 21-31, February.
    2. Ignacio Palacios-Huerta & Oscar Volij, 2004. "The Measurement of Intellectual Influence," Econometrica, Econometric Society, vol. 72(3), pages 963-977, May.
    3. Richard Dusansky & Clayton J. Vernon, 1998. "Rankings of U.S. Economics Departments," Journal of Economic Perspectives, American Economic Association, vol. 12(1), pages 157-170, Winter.
    4. Kristie M. Engemann & Howard J. Wall, 2009. "A journal ranking for the ambitious economist," Review, Federal Reserve Bank of St. Louis, vol. 91(May), pages 127-140.
    5. Kodrzycki Yolanda K. & Yu Pingkang, 2006. "New Approaches to Ranking Economics Journals," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 5(1), pages 1-44, August.
    6. Kalaitzidakis, P. & Mamuneas, T.P. & Stengos, T., 2003. "Rankings of Academic Journals and Institutions," Working Papers 2003-8, University of Guelph, Department of Economics and Finance.
    7. Liebowitz, S J & Palmer, J P, 1984. "Assessing the Relative Impacts of Economic Journals," Journal of Economic Literature, American Economic Association, vol. 22(1), pages 77-88, March.
    8. Pantelis Kalaitzidakis & Theofanis P. Mamuneas & Thanasis Stengos, 2003. "Rankings of Academic Journals and Institutions in Economics," Journal of the European Economic Association, MIT Press, vol. 1(6), pages 1346-1366, December.
    9. Yolanda Kodrzycki & Pingkang David Yu, 2005. "New approaches to ranking economics journals," Working Papers 05-12, Federal Reserve Bank of Boston.
    10. Tibor Braun & Wolfgang Glänzel & András Schubert, 2006. "A Hirsch-type index for journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 69(1), pages 169-173, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    RePEc Biblio mentions

    As found on the RePEc Biblio, the curated bibliography for Economics:
    1. > Economics Profession > Ranking in Economics > Ranking Journals

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Miguel Sarmiento, 2010. "Central Bank Economic Research: Output, Demand, Productivity, and Relevance," Money Affairs, CEMLA, vol. 0(2), pages 211-240, July-Dece.
    2. repec:ner:leuven:urn:hdl:123456789/327130 is not listed on IDEAS
    3. Anne E Winkler & Sharon G Levin & Paula E Stephan & Wolfgang Gl&aauml;nzel, 2014. "Publishing Trends in Economics across Colleges and Universities, 1991–2007," Eastern Economic Journal, Palgrave Macmillan;Eastern Economic Association, vol. 40(4), pages 560-582, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. David Anderson & John Tressler, 2008. "Research output in New Zealand economics departments 2000-2006: A stock approach," New Zealand Economic Papers, Taylor & Francis Journals, vol. 42(2), pages 155-189.
    2. David L. Anderson & John Tressler, 2010. "The Merits of Using Citation-Based Journal Weighting Schemes to Measure Research Performance in Economics: The Case of New Zealand," Working Papers in Economics 10/03, University of Waikato.
    3. Pantelis Kalaitzidakis & Theofanis P. Mamuneas & Thanasis Stengos, 2011. "An updated ranking of academic journals in economics," Canadian Journal of Economics, Canadian Economics Association, vol. 44(4), pages 1525-1538, November.
    4. David L. Anderson & John Tressler, 2013. "The Relevance of the “h-” and “g-” Index to Economics in the Context of A Nation-Wide Research Evaluation Scheme: The New Zealand Case," Economic Papers, The Economic Society of Australia, vol. 32(1), pages 81-94, March.
    5. Yong Bao & Melody Lo & Franklin G. Mixon, 2010. "General-interest versus specialty journals: Using intellectual influence of econometrics research to rank economics journals and articles," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(2), pages 345-353.
    6. Wohlrabe, Klaus, 2016. "Taking the Temperature: A Meta-Ranking of Economics Journals," MPRA Paper 68933, University Library of Munich, Germany.
    7. Chagas , André Luis Squarize, 2017. "Publish or Perish: um ranking de revistas da subárea de Economia Regional e Urbana para os pesquisadores brasileiros," Revista Brasileira de Estudos Regionais e Urbanos, Associação Brasileira de Estudos Regionais e Urbanos (ABER), vol. 11(4), pages 515-536.
    8. Yu, Xiaohua & Gao, Zhifeng, 2010. "An updated ranking of the economic research institutions in China (2000-2009)," China Economic Review, Elsevier, vol. 21(4), pages 571-581, December.
    9. J. A. García & Rosa Rodriguez-Sánchez & J. Fdez-Valdivia & J. Martinez-Baena, 2012. "On first quartile journals which are not of highest impact," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(3), pages 925-943, March.
    10. David L. Anderson & John Tressler, 2009. "The Excellence in Research for Australia Scheme: An Evaluation of the Draft Journal Weights for Economics," Working Papers in Economics 09/07, University of Waikato.
    11. Koczy, L. & Nichifor, A. & Strobel, M., 2010. "Intellectual Influence: quality versus quantity," Research Memorandum 029, Maastricht University, Maastricht Research School of Economics of Technology and Organization (METEOR).
    12. J. A. García & Rosa Rodriguez-Sánchez & J. Fdez-Valdivia, 2011. "Overall prestige of journals with ranking score above a given threshold," Scientometrics, Springer;Akadémiai Kiadó, vol. 89(1), pages 229-243, October.
    13. Laszlo A. Koczy & Martin Strobel, 2010. "The World Cup of Economics Journals: A Ranking by a Tournament Method," CERS-IE WORKING PAPERS 1018, Institute of Economics, Centre for Economic and Regional Studies.
    14. Johannes König & David I. Stern & Richard S.J. Tol, 2022. "Confidence Intervals for Recursive Journal Impact Factors," Tinbergen Institute Discussion Papers 22-038/III, Tinbergen Institute.
    15. David I. Stern, 2013. "Uncertainty Measures for Economics Journal Impact Factors," Journal of Economic Literature, American Economic Association, vol. 51(1), pages 173-189, March.
    16. David L. Anderson & John Tressler, 2008. "Research Output in New Zealand Economics Department 2000-2006," Working Papers in Economics 08/05, University of Waikato.
    17. Ham, John C. & Wright, Julian & Ye, Ziqiu, 2023. "Documenting and Explaining the Dramatic Rise of the New Society Journals in Economics," IZA Discussion Papers 16337, Institute of Labor Economics (IZA).
    18. Javier Ruiz-Castillo, 2008. "Economics research in Spain during the 1990s: a literature review," Spanish Economic Review, Springer;Spanish Economic Association, vol. 10(3), pages 221-249, September.
    19. George Emm Halkos & Nickolaos G. Tzeremes, 2011. "Measuring economic journals’ citation efficiency: a data envelopment analysis approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(3), pages 979-1001, September.
    20. Chan, Ho Fai & Frey, Bruno S. & Gallus, Jana & Torgler, Benno, 2014. "Academic honors and performance," Labour Economics, Elsevier, vol. 31(C), pages 188-204.

    More about this item

    Keywords

    Economics;

    NEP fields

    This paper has been announced in the following NEP Reports:

    Lists

    This item is featured on the following reading lists, Wikipedia, or ReplicationWiki pages:
    1. Rankings of Economists, Economics Departments and Economics Journals

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:fip:fedlwp:2009-014. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Anna Oates (email available below). General contact details of provider: https://edirc.repec.org/data/frbslus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.