IDEAS home Printed from https://ideas.repec.org/p/een/crwfrp/1302.html
   My bibliography  Save this paper

Uncertainty Measures for Economics Journal Impact Factors

Author

Listed:
  • David I. Stern

    (Crawford School of Public Policy, The Australian National University Author-Email: david.stern@anu.edu.au)

Abstract

Academic economists appear to be intensely interested in rankings of journals, institutions, and individuals. Yet there is little discussion of the uncertainty associated with these rankings. To illustrate the uncertainty associated with citations-based rankings, I compute the standard error of the impact factor for all economics journals with a five-year impact factor in the 2011 Journal Citations Report. I use these to derive confidence intervals for the impact factors as well as ranges of possible rank for a subset of thirty journals. I find that the impact factors of the top two journals are well defined and set these journals apart in a clearly defined group. An elite group of 9-11 mainstream journals can also be fairly reliably distinguished. The four bottom ranked journals are also fairly clearly set apart. For the remainder of the distribution, confidence intervals overlap and rankings are quite uncertain.

Suggested Citation

  • David I. Stern, 2013. "Uncertainty Measures for Economics Journal Impact Factors," Crawford School Research Papers 1302, Crawford School of Public Policy, The Australian National University.
  • Handle: RePEc:een:crwfrp:1302
    as

    Download full text from publisher

    File URL: http://ssrn.com/abstract=2199162
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. Christian Zimmermann, 2013. "Academic Rankings with RePEc," Econometrics, MDPI, Open Access Journal, vol. 1(3), pages 1-32, December.
    2. David N. Laband & Robert D. Tollison, 2003. "Dry Holes in Economic Research," Kyklos, Wiley Blackwell, vol. 56(2), pages 161-173, May.
    3. S. Redner, 1998. "How popular is your paper? An empirical study of the citation distribution," The European Physical Journal B: Condensed Matter and Complex Systems, Springer;EDP Sciences, vol. 4(2), pages 131-134, July.
    4. repec:spr:scient:v:78:y:2009:i:2:d:10.1007_s11192-008-2166-9 is not listed on IDEAS
    5. Kalaitzidakis, P. & Mamuneas, T.P. & Stengos, T., 2003. "Rankings of Academic Journals and Institutions," Working Papers 2003-8, University of Guelph, Department of Economics and Finance.
    6. Ma, Chunbo & Stern, David I., 2006. "Environmental and ecological economics: A citation analysis," Ecological Economics, Elsevier, vol. 58(3), pages 491-506, June.
    7. repec:spr:scient:v:51:y:2001:i:2:d:10.1023_a:1012705818635 is not listed on IDEAS
    8. Andrew J. Oswald, 2007. "An Examination of the Reliability of Prestigious Scholarly Journals: Evidence and Implications for Decision-Makers," Economica, London School of Economics and Political Science, vol. 74(293), pages 21-31, February.
    9. Burton, M P & Phimister, Euan, 1995. "Core Journals: A Reappraisal of the Diamond List," Economic Journal, Royal Economic Society, vol. 105(429), pages 361-373, March.
    10. Laband, David N & Piette, Michael J, 1994. "The Relative Impacts of Economics Journals: 1970-1990," Journal of Economic Literature, American Economic Association, vol. 32(2), pages 640-666, June.
    11. Liebowitz, S J & Palmer, J P, 1984. "Assessing the Relative Impacts of Economic Journals," Journal of Economic Literature, American Economic Association, vol. 22(1), pages 77-88, March.
    12. George Emm Halkos & Nickolaos G. Tzeremes, 2011. "Measuring economic journals’ citation efficiency: a data envelopment analysis approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(3), pages 979-1001, September.
    13. Urs Fischbacher, 2007. "z-Tree: Zurich toolbox for ready-made economic experiments," Experimental Economics, Springer;Economic Science Association, vol. 10(2), pages 171-178, June.
    14. repec:spr:scient:v:92:y:2012:i:2:d:10.1007_s11192-012-0679-8 is not listed on IDEAS
    15. Richard Dusansky & Clayton J. Vernon, 1998. "Rankings of U.S. Economics Departments," Journal of Economic Perspectives, American Economic Association, vol. 12(1), pages 157-170, Winter.
    16. repec:spr:scient:v:92:y:2012:i:2:d:10.1007_s11192-012-0634-8 is not listed on IDEAS
    17. repec:spr:scient:v:92:y:2012:i:2:d:10.1007_s11192-012-0747-0 is not listed on IDEAS
    18. Tom Coupé, 2003. "Revealed Performances: Worldwide Rankings of Economists and Economics Departments, 1990-2000," Journal of the European Economic Association, MIT Press, vol. 1(6), pages 1309-1345, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Blog mentions

    As found by EconAcademics.org, the blog aggregator for Economics research:
    1. Uncertainty Measures for Economics Journal Impact Factors
      by David Stern in Stochastic Trend on 2013-01-11 14:37:00

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Martin Richards, 2013. "Putting the blue pencil away and taking down the name board," Transportation, Springer, vol. 40(6), pages 1087-1104, November.
    2. Wohlrabe, Klaus & Bornmann, Lutz, 2017. "Normalization of citation impact in economics," MPRA Paper 80384, University Library of Munich, Germany.
    3. Wohlrabe, Klaus, 2016. "Taking the Temperature: A Meta-Ranking of Economics Journals," MPRA Paper 68933, University Library of Munich, Germany.
    4. Seiler, Christian & Wohlrabe, Klaus, 2014. "How robust are journal rankings based on the impact factor? Evidence from the economic sciences," Journal of Informetrics, Elsevier, vol. 8(4), pages 904-911.
    5. repec:eee:infome:v:11:y:2017:i:3:p:689-703 is not listed on IDEAS
    6. Jakob Kapeller & Matthias Aistleitner & Stefan Steinerberger, 2017. "Citation Patterns in Economics and Beyond: Assessing the Peculiarities of Economics from Two Scientometric Perspectives," ICAE Working Papers 60, Johannes Kepler University, Institute for Comprehensive Analysis of the Economy.
    7. Victoria Anauati & Sebastian Galiani & Ramiro H. Gálvez, 2016. "Quantifying The Life Cycle Of Scholarly Articles Across Fields Of Economic Research," Economic Inquiry, Western Economic Association International, vol. 54(2), pages 1339-1355, April.
    8. Bertocchi, Graziella & Gambardella, Alfonso & Jappelli, Tullio & Nappi, Carmela A. & Peracchi, Franco, 2015. "Bibliometric evaluation vs. informed peer review: Evidence from Italy," Research Policy, Elsevier, vol. 44(2), pages 451-466.
    9. Lehmann, Robert & Wohlrabe, Klaus, 2017. "Who is the ‘Journal Grand Master’? A new ranking based on the Elo rating system," Journal of Informetrics, Elsevier, vol. 11(3), pages 800-809.
    10. Lea Kosnik, 2014. "What Have Economists Been Doing for the Last 50 Years? A Text Analysis of Published Academic Research from 1960-2010," Working Papers 1004, University of Missouri-St. Louis, Department of Economics, revised Apr 2015.
    11. repec:aea:jeclit:v:56:y:2018:i:1:p:115-56 is not listed on IDEAS
    12. David L. Anderson & John Tressler, 2016. "Citation-Capture Rates for Economics Journals: Do they Differ from Other Disciplines and Does it Matter?," Economic Papers, The Economic Society of Australia, vol. 35(1), pages 73-85, March.
    13. Kosnik, Lea-Rachel, 2015. "What have economists been doing for the last 50 years? A text analysis of published academic research from 1960-2010," Economics - The Open-Access, Open-Assessment E-Journal, Kiel Institute for the World Economy (IfW), vol. 9, pages 1-38.
    14. Daniel S. Hamermesh, 2015. "Citations in Economics: Measurement, Uses and Impacts," NBER Working Papers 21754, National Bureau of Economic Research, Inc.
    15. repec:spr:scient:v:113:y:2017:i:3:d:10.1007_s11192-017-2507-7 is not listed on IDEAS
    16. Imad A. Moosa, 2016. "A Critique of the Bucket Classification of Journals: The ABDC List as an Example," The Economic Record, The Economic Society of Australia, vol. 92(298), pages 448-463, September.
    17. repec:spr:scient:v:111:y:2017:i:3:d:10.1007_s11192-017-2351-9 is not listed on IDEAS
    18. William Horrace & Christopher Parmeter, 2016. "Accounting for Multiplicity in Inference on Economics Journal Rankings," Working Papers 2016-08, University of Miami, Department of Economics.
    19. repec:spr:scient:v:105:y:2015:i:2:d:10.1007_s11192-015-1747-7 is not listed on IDEAS
    20. Lutz Bornmann & Alexander Butz & Klaus Wohlrabe, 2018. "What are the top five journals in economics? A new meta-ranking," Applied Economics, Taylor & Francis Journals, vol. 50(6), pages 659-675, February.
    21. repec:spr:scient:v:113:y:2017:i:1:d:10.1007_s11192-017-2471-2 is not listed on IDEAS
    22. Kosteas, Vasilios D., 2015. "Journal impact factors and month of publication," Economics Letters, Elsevier, vol. 135(C), pages 77-79.

    More about this item

    Keywords

    academic; journals; economics;

    JEL classification:

    • A14 - General Economics and Teaching - - General Economics - - - Sociology of Economics

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:een:crwfrp:1302. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (David Stern). General contact details of provider: http://edirc.repec.org/data/asanuau.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.