IDEAS home Printed from https://ideas.repec.org/a/bla/jecsur/v34y2020i4p876-921.html
   My bibliography  Save this article

Grading Journals In Economics: The Abcs Of The Abdc

Author

Listed:
  • Joseph Gerald Hirschberg
  • Jeanette Ngaire Lye

Abstract

Many institutions and governments grade academic journals for the evaluation of research. In this paper, we implement a multi‐bibliometric methodology for the evaluation of such a list of journal grades. We examine the grades assigned by the Australian Business Deans Council (ABDC) for over 750 journals in the fields of economics and statistics. Firstly, we generate up to 48 bibliometric‐based grades for each journal based on the grade distribution implied by the ABDC. Secondly, we categorize the bibliometrics employing a cluster analysis of an interrater agreement statistic. Thirdly, we present a visualization of the consistency of the grading by journal. Finally, we list those journals where the majority of the matched bibliometrics indicate a higher or lower grade than their ABDC grade.

Suggested Citation

  • Joseph Gerald Hirschberg & Jeanette Ngaire Lye, 2020. "Grading Journals In Economics: The Abcs Of The Abdc," Journal of Economic Surveys, Wiley Blackwell, vol. 34(4), pages 876-921, September.
    • Handle: RePEc:bla:jecsur:v:34:y:2020:i:4:p:876-921
    DOI: 10.1111/joes.12369
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/joes.12369
    Download Restriction: no
    File URL: https://libkey.io/10.1111/joes.12369?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Christian Zimmermann, 2013. "Academic Rankings with RePEc," Econometrics, MDPI, vol. 1(3), pages 1-32, December.
    2. Lutz Bornmann & Alexander Butz & Klaus Wohlrabe, 2018. "What are the top five journals in economics? A new meta-ranking," Applied Economics, Taylor & Francis Journals, vol. 50(6), pages 659-675, February.
    3. Stock, James H. & Watson, Mark W., 2014. "Estimating turning points using large data sets," Journal of Econometrics, Elsevier, vol. 178(P2), pages 368-381.
    4. Charles E. Scott & John J. Siegfried, 2019. "American Economic Association Universal Academic Questionnaire Summary Statistics," AEA Papers and Proceedings, American Economic Association, vol. 109, pages 590-592, May.
    5. Stan J. Liebowitz, 2014. "Willful Blindness: The Inefficient Reward Structure In Academic Research," Economic Inquiry, Western Economic Association International, vol. 52(4), pages 1267-1283, October.
    6. Mingers, John & Yang, Liying, 2017. "Evaluating journal quality: A review of journal citation indicators and ranking in business and management," European Journal of Operational Research, Elsevier, vol. 257(1), pages 323-337.
    7. Wang, Jian & Veugelers, Reinhilde & Stephan, Paula, 2017. "Bias against novelty in science: A cautionary tale for users of bibliometric indicators," Research Policy, Elsevier, vol. 46(8), pages 1416-1436.
    8. Chia-Lin Chang & Michael McAleer & Les Oxley, 2011. "What makes a great journal great in the sciences? Which came first, the chicken or the egg?," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(1), pages 17-40, April.
    9. Seiler, Christian & Wohlrabe, Klaus, 2012. "Ranking economists on the basis of many indicators: An alternative approach using RePEc data," Journal of Informetrics, Elsevier, vol. 6(3), pages 389-402.
    10. Mikko Packalen & Jay Bhattacharya, 2017. "Neophilia ranking of scientific journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(1), pages 43-64, January.
    11. Heather Piwowar, 2013. "Value all research products," Nature, Nature, vol. 493(7431), pages 159-159, January.
    12. Imad Moosa, 2011. "The demise of the ARC journal ranking scheme: an ex post analysis of the accounting and finance journals," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 51(3), pages 809-836, September.
    13. Glenn Ellison, 2013. "How Does the Market Use Citation Data? The Hirsch Index in Economics," American Economic Journal: Applied Economics, American Economic Association, vol. 5(3), pages 63-90, July.
    14. Hirschberg, Joseph G. & Maasoumi, Esfandiar & Slottje, Daniel J., 1991. "Cluster analysis for measuring welfare and quality of life across countries," Journal of Econometrics, Elsevier, vol. 50(1-2), pages 131-150, October.
    15. Watson, Mark W. & Stock, James H., 2014. "Estimating turning points using large data sets," Scholarly Articles 33192198, Harvard University Department of Economics.
    16. David I. Stern, 2013. "Uncertainty Measures for Economics Journal Impact Factors," Journal of Economic Literature, American Economic Association, vol. 51(1), pages 173-189, March.
    17. Mario Biagioli, 2016. "Watch out for cheats in citation game," Nature, Nature, vol. 535(7611), pages 201-201, July.
    18. Chia‐Lin Chang & Michael McAleer & Les Oxley, 2011. "What Makes A Great Journal Great In Economics? The Singer Not The Song," Journal of Economic Surveys, Wiley Blackwell, vol. 25(2), pages 326-361, April.
    19. Luca Pratelli & Alberto Baccini & Lucio Barabesi & Marzia Marcheselli, 2012. "Statistical Analysis of the Hirsch Index," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 39(4), pages 681-694, December.
    20. Haucap, Justus & Thomas, Tobias & Wohlrabe, Klaus, 2017. "Publication performance vs. influence: On the questionable value of quality weighted publication rankings," DICE Discussion Papers 277, Heinrich Heine University Düsseldorf, Düsseldorf Institute for Competition Economics (DICE).
    21. Klaus Wohlrabe & Elisabeth Friedrich, 2017. "The efficiency of economics departments reconsidered," Economics Bulletin, AccessEcon, vol. 37(3), pages 1602-1611.
    22. László Kóczy & Alexandru Nichifor, 2013. "The intellectual influence of economic journals: quality versus quantity," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 52(3), pages 863-884, April.
    23. Motty Perry & Philip J. Reny, 2016. "How to Count Citations If You Must," American Economic Review, American Economic Association, vol. 106(9), pages 2722-2741, September.
    24. Imad A. Moosa, 2016. "A Critique of the Bucket Classification of Journals: The ABDC List as an Example," The Economic Record, The Economic Society of Australia, vol. 92(298), pages 448-463, September.
    25. Slottje, Daniel J, 1991. "Measuring the Quality of Life across Countries," The Review of Economics and Statistics, MIT Press, vol. 73(4), pages 684-693, November.
    26. Daniel S. Hamermesh, 2018. "Citations in Economics: Measurement, Uses, and Impacts," Journal of Economic Literature, American Economic Association, vol. 56(1), pages 115-156, March.
    27. David N. Laband, 2013. "On the Use and Abuse of Economics Journal Rankings," Economic Journal, Royal Economic Society, vol. 0, pages 223-254, August.
    28. Laszlo A. Koczy & Martin Strobel, 2010. "The World Cup of Economics Journals: A Ranking by a Tournament Method," CERS-IE WORKING PAPERS 1018, Institute of Economics, Centre for Economic and Regional Studies.
    29. David Card & Stefano DellaVigna, 2013. "Nine Facts about Top Journals in Economics," Journal of Economic Literature, American Economic Association, vol. 51(1), pages 144-161, March.
    30. Stefanie Haustein & Isabella Peters & Judit Bar-Ilan & Jason Priem & Hadas Shema & Jens Terliesner, 2014. "Coverage and adoption of altmetrics sources in the bibliometric community," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1145-1163, November.
    31. Zohreh Zahedi & Rodrigo Costas & Paul Wouters, 2014. "How well developed are altmetrics? A cross-disciplinary analysis of the presence of ‘alternative metrics’ in scientific publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1491-1513, November.
    32. Howard Bodenhorn, 2003. "Economic Scholarship at Elite Liberal Arts Colleges: A Citation Analysis with Rankings," The Journal of Economic Education, Taylor & Francis Journals, vol. 34(4), pages 341-359, December.
    33. Pierre-Philippe Combes & Laurent Linnemer, 2010. "Inferring Missing Citations: A Quantitative Multi-Criteria Ranking of all Journals in Economics," Working Papers halshs-00520325, HAL.
    34. David W. Nickerson & Todd Rogers, 2014. "Political Campaigns and Big Data," Journal of Economic Perspectives, American Economic Association, vol. 28(2), pages 51-74, Spring.
    35. Tombazos, Christis G. & Dobra, Matthew, 2014. "Formulating research policy on expert advice," European Economic Review, Elsevier, vol. 72(C), pages 166-181.
    36. Rodrigo Costas & Zohreh Zahedi & Paul Wouters, 2015. "Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(10), pages 2003-2019, October.
    37. Daniel S. Hamermesh & Gerard A. Pfann, 2012. "Reputation And Earnings: The Roles Of Quality And Quantity In Academe," Economic Inquiry, Western Economic Association International, vol. 50(1), pages 1-16, January.
    38. Bornmann, Lutz, 2014. "Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics," Journal of Informetrics, Elsevier, vol. 8(4), pages 895-903.
    39. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Domingo Docampo & Vicente Safón, 2021. "Journal ratings: a paper affiliation methodology," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 8063-8090, September.
    2. Yihui Lan & Kenneth W Clements & Zong Ken Chai, 2022. "Australian PhDs in Economics and Finance: Professional Activities, Productivity and Prospects," Economics Discussion / Working Papers 22-04, The University of Western Australia, Department of Economics.
    3. Buckle, Robert A & Creedy, John, 2022. "The Performance Based Research Fund in NZ: Taking Stock and Looking Forward," Working Paper Series 21354, Victoria University of Wellington, Chair in Public Finance.
    4. Yihui Lan & Kenneth W. Clements & Zong Ken Chai, 2023. "How Productive Are Economics and Finance PhDs?," Australian Economic Review, The University of Melbourne, Melbourne Institute of Applied Economic and Social Research, vol. 56(4), pages 442-461, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lutz Bornmann & Klaus Wohlrabe, 2019. "Normalisation of citation impact in economics," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(2), pages 841-884, August.
    2. Lutz Bornmann & Alexander Butz & Klaus Wohlrabe, 2018. "What are the top five journals in economics? A new meta-ranking," Applied Economics, Taylor & Francis Journals, vol. 50(6), pages 659-675, February.
    3. Daniel S. Hamermesh, 2018. "Citations in Economics: Measurement, Uses, and Impacts," Journal of Economic Literature, American Economic Association, vol. 56(1), pages 115-156, March.
    4. David L. Anderson & John Tressler, 2017. "Researcher rank stability across alternative output measurement schemes in the context of a time limited research evaluation: the New Zealand case," Applied Economics, Taylor & Francis Journals, vol. 49(45), pages 4542-4553, September.
    5. Wohlrabe, Klaus, 2016. "Taking the Temperature: A Meta-Ranking of Economics Journals," MPRA Paper 68933, University Library of Munich, Germany.
    6. J. Atsu Amegashie, 2020. "Citations And Incentives In Academic Contests," Economic Inquiry, Western Economic Association International, vol. 58(3), pages 1233-1244, July.
    7. María Victoria Anauati & Sebastian Galiani & Ramiro H. Gálvez, 2020. "Differences In Citation Patterns Across Journal Tiers: The Case Of Economics," Economic Inquiry, Western Economic Association International, vol. 58(3), pages 1217-1232, July.
    8. Koffi, Marlene, 2021. "Innovative ideas and gender inequality," CLEF Working Paper Series 35, Canadian Labour Economics Forum (CLEF), University of Waterloo.
    9. Tahamtan, Iman & Bornmann, Lutz, 2018. "Creativity in science and the link to cited references: Is the creative potential of papers reflected in their cited references?," Journal of Informetrics, Elsevier, vol. 12(3), pages 906-930.
    10. Matthias Aistleitner & Jakob Kapeller & Stefan Steinerberger, 2018. "Citation Patterns in Economics and Beyond," Working Papers Series 85, Institute for New Economic Thinking.
    11. Syed Hasan & Robert Breunig, 2021. "Article length and citation outcomes," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 7583-7608, September.
    12. María Victoria Anauati & Sebastian Galiani & Ramiro H. Gálvez, 2018. "Differences in citation patterns across journal tiers in economics," Documentos de Trabajo 16701, The Latin American and Caribbean Economic Association (LACEA).
    13. David L. Anderson & John Tressler, 2015. "Are Researcher Rankings Stable Across Alternative Output Measurement Schemes in the Context of a Time Limited Research Evaluation? The New Zealand Case," Working Papers in Economics 15/10, University of Waikato.
    14. James J. Heckman & Sidharth Moktan, 2020. "Publishing and promotion in economics - The tyranny of the Top Five," Vox eBook Chapters, in: Sebastian Galliani & Ugo Panizza (ed.), Publishing and Measuring Success in Economics, edition 1, volume 1, chapter 1, pages 23-32, Centre for Economic Policy Research.
    15. Victoria Anauati & Sebastian Galiani & Ramiro H. Gálvez, 2016. "Quantifying The Life Cycle Of Scholarly Articles Across Fields Of Economic Research," Economic Inquiry, Western Economic Association International, vol. 54(2), pages 1339-1355, April.
    16. Sergio Copiello, 2020. "Other than detecting impact in advance, alternative metrics could act as early warning signs of retractions: tentative findings of a study into the papers retracted by PLoS ONE," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2449-2469, December.
    17. Yu Liu & Dan Lin & Xiujuan Xu & Shimin Shan & Quan Z. Sheng, 2018. "Multi-views on Nature Index of Chinese academic institutions," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 823-837, March.
    18. John Gibson & David L. Anderson & John Tressler, 2017. "Citations Or Journal Quality: Which Is Rewarded More In The Academic Labor Market?," Economic Inquiry, Western Economic Association International, vol. 55(4), pages 1945-1965, October.
    19. Claude Diebolt & Michael Haupert, 2021. "The Role of Cliometrics in History and Economics," Working Papers of BETA 2021-26, Bureau d'Economie Théorique et Appliquée, UDS, Strasbourg.
    20. David L. Anderson & John Tressler, 2016. "Citation-Capture Rates for Economics Journals: Do they Differ from Other Disciplines and Does it Matter?," Economic Papers, The Economic Society of Australia, vol. 35(1), pages 73-85, March.

    More about this item

    JEL classification:

    • C49 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - Other
    • O30 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - General
    • Y10 - Miscellaneous Categories - - Data: Tables and Charts - - - Data: Tables and Charts

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jecsur:v:34:y:2020:i:4:p:876-921. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0950-0804 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.