IDEAS home Printed from https://ideas.repec.org/p/iza/izadps/dp2070.html
   My bibliography  Save this paper

An Examination of the Reliability of Prestigious Scholarly Journals: Evidence and Implications for Decision-makers

Author

Listed:
  • Oswald, Andrew J.

    (University of Warwick)

Abstract

In universities all over the world, hiring and promotion committees regularly hear the argument: “this is important work because it is about to appear in prestigious journal X”. Moreover, those who allocate levels of research funding, such as in the multi-billion pound Research Assessment Exercise in UK universities, often come under pressure to assess research quality in a mechanical way by using journal prestige ratings. The results in this paper suggest that such tendencies are dangerous. It uses total citations over a quarter of a century as the criterion. The paper finds that it is far better to publish the best article in an issue of a medium-quality journal like the Oxford Bulletin of Economics and Statistics than to publish the worst article (or often the worst 4 articles) in an issue of a top journal like the American Economic Review. Implications are discussed.

Suggested Citation

  • Oswald, Andrew J., 2006. "An Examination of the Reliability of Prestigious Scholarly Journals: Evidence and Implications for Decision-makers," IZA Discussion Papers 2070, Institute of Labor Economics (IZA).
  • Handle: RePEc:iza:izadps:dp2070
    as

    Download full text from publisher

    File URL: https://docs.iza.org/dp2070.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Jerry G. Thursby, 2000. "What Do We Say about Ourselves and What Does It Mean? Yet Another Look at Economics Department Research," Journal of Economic Literature, American Economic Association, vol. 38(2), pages 383-404, June.
    2. Anthony F J van Raan, 1998. "Assessment of social sciences: The use of advanced bibliometric methods as a necessary complement of peer review," Research Evaluation, Oxford University Press, vol. 7(1), pages 2-6, April.
    3. Nancy K. Bayers, 2005. "Using ISI data in the analysis of German national and institutional research output," Scientometrics, Springer;Akadémiai Kiadó, vol. 62(1), pages 155-163, January.
    4. David N. Laband, 1990. "Is There Value-Added from the Review Process in Economics?: Preliminary Evidence from Authors," The Quarterly Journal of Economics, Oxford University Press, vol. 105(2), pages 341-352.
    5. Moore, William J & Newman, Robert J & Turnbull, Geoffrey K, 1998. "Do Academic Salaries Decline with Seniority?," Journal of Labor Economics, University of Chicago Press, vol. 16(2), pages 352-366, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Michael J. Hilmer & Christiana E. Hilmer, 2009. "Fishes, Ponds, And Productivity: Student‐Advisor Matching And Early Career Publishing Success For Economics Phds," Economic Inquiry, Western Economic Association International, vol. 47(2), pages 290-303, April.
    2. Daniel S. Hamermesh, 2018. "Citations in Economics: Measurement, Uses, and Impacts," Journal of Economic Literature, American Economic Association, vol. 56(1), pages 115-156, March.
    3. Goodall, Amanda H., 2009. "Highly cited leaders and the performance of research universities," Research Policy, Elsevier, vol. 38(7), pages 1079-1092, September.
    4. Justus Haucap & Johannes Muck, 2015. "What drives the relevance and reputation of economics journals? An update from a survey among economists," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(3), pages 849-877, June.
    5. Rolf Ketzler & Klaus F. Zimmermann, 2009. "Publications: German economic research institutes on track," Scientometrics, Springer;Akadémiai Kiadó, vol. 80(1), pages 231-252, July.
    6. Mehdi Rhaiem & Nabil Amara, 2020. "Determinants of research efficiency in Canadian business schools: evidence from scholar-level data," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 53-99, October.
    7. Bruno Frey, 2005. "Problems with Publishing: Existing State and Solutions," European Journal of Law and Economics, Springer, vol. 19(2), pages 173-190, April.
    8. Valadkhani, Abbas & Ville, Simon, 2008. "Identifying the Most Research Intensive Faculties of Business in Australia: A Multidimensional Approach," Economics Working Papers wp08-03, School of Economics, University of Wollongong, NSW, Australia.
    9. Ofer H. Azar, 2006. "The Academic Review Process: How Can We Make it More Efficient?," The American Economist, Sage Publications, vol. 50(1), pages 37-50, March.
    10. Di Vaio, Gianfranco & Waldenström, Daniel & Weisdorf, Jacob, 2012. "Citation success: Evidence from economic history journal publications," Explorations in Economic History, Elsevier, vol. 49(1), pages 92-104.
    11. Jerome K. Vanclay, 2012. "Impact factor: outdated artefact or stepping-stone to journal certification?," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 211-238, August.
    12. Fabel, Oliver & Lehmann, Erik & Warning, Susanne, 2002. "Vorträge als Qualitätsindikator: Empirische Evidenz der Jahrestagungen des Vereins für Socialpolitik," Discussion Papers, Series I 321, University of Konstanz, Department of Economics.
    13. Azar, Ofer H., 2008. "Evolution of social norms with heterogeneous preferences: A general model and an application to the academic review process," Journal of Economic Behavior & Organization, Elsevier, vol. 65(3-4), pages 420-435, March.
    14. Philippe Jeannin, 2004. "Les économistes et leurs revues," Revue d'économie politique, Dalloz, vol. 114(3), pages 275-288.
    15. William J. Moore & Robert J. Newman & Peter J. Sloane & Jeremy D. Steely, 2002. "Productivity Effects of Research Assessment Exercises," Departmental Working Papers 2002-15, Department of Economics, Louisiana State University.
    16. Juho Jokinen & Jaakko Pehkonen, 2017. "Promotions and Earnings – Gender or Merit? Evidence from Longitudinal Personnel Data," Journal of Labor Research, Springer, vol. 38(3), pages 306-334, September.
    17. Glenn Ellison, 2002. "The Slowdown of the Economics Publishing Process," Journal of Political Economy, University of Chicago Press, vol. 110(5), pages 947-993, October.
    18. Hadavand, Aboozar & Hamermesh, Daniel S. & Wilson, Wesley W., 2019. "Is Scholarly Refereeing Productive (at the Margin)?," IZA Discussion Papers 12866, Institute of Labor Economics (IZA).
    19. Barrett, Christopher B. & Bailey, DeeVon, 1999. "Are Agricultural Experiment Station Faculty Salaries Competitively or Monopsonistically Determined?," Agricultural and Resource Economics Review, Cambridge University Press, vol. 28(1), pages 1-10, April.
    20. David L. Anderson & John Tressler, 2011. "Ranking Economics Departments In Terms Of Residual Productivity: New Zealand Economics Departments, 2000–2006," Australian Economic Papers, Wiley Blackwell, vol. 50(4), pages 157-168, December.

    More about this item

    Keywords

    economics journals; research productivity; citations; Research Assessment Exercise;
    All these keywords.

    JEL classification:

    • A11 - General Economics and Teaching - - General Economics - - - Role of Economics; Role of Economists
    • O3 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp2070. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Holger Hinte (email available below). General contact details of provider: https://edirc.repec.org/data/izaaade.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.