IDEAS home Printed from https://ideas.repec.org/a/bla/ecinqu/v55y2017i4p1945-1965.html
   My bibliography  Save this article

Citations Or Journal Quality: Which Is Rewarded More In The Academic Labor Market?

Author

Listed:
  • John Gibson
  • David L. Anderson
  • John Tressler

Abstract

Research quality can be evaluated using citations or from the prestige of the journal that publishes the research. Recent studies advocate for more weight on citations, which measure actual impact, while the journal where an article publishes is merely a predictor of whether it was thought likely to have an impact. Yet there is little comprehensive evidence on the role of citations versus journal quality in evaluating research. In this paper we use data on tenured economists in the University of California system to relate their salary to their lifetime publications of 5500 articles in almost 700 different academic journals and to the 140,000 citations to these articles. The results show little role for citations in affecting faculty salary, with an impact only one-seventh that of a measure of journal publications. The distribution of citations, whether using an h-index or the generalized h-index proposed by Ellison (2013), is also not a significant predictor of salary.
(This abstract was borrowed from another version of this item.)

Suggested Citation

  • John Gibson & David L. Anderson & John Tressler, 2017. "Citations Or Journal Quality: Which Is Rewarded More In The Academic Labor Market?," Economic Inquiry, Western Economic Association International, vol. 55(4), pages 1945-1965, October.
    • Handle: RePEc:bla:ecinqu:v:55:y:2017:i:4:p:1945-1965
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1111/ecin.12455
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Glenn Ellison, 2013. "How Does the Market Use Citation Data? The Hirsch Index in Economics," American Economic Journal: Applied Economics, American Economic Association, vol. 5(3), pages 63-90, July.
    2. Pantelis Kalaitzidakis & Theofanis P. Mamuneas & Thanasis Stengos, 2011. "An updated ranking of academic journals in economics," Canadian Journal of Economics, Canadian Economics Association, vol. 44(4), pages 1525-1538, November.
    3. Daniel Sgroi & Andrew J. Oswald, 2013. "How Should Peer‐review Panels Behave?," Economic Journal, Royal Economic Society, vol. 0, pages 255-278, August.
    4. Kodrzycki Yolanda K. & Yu Pingkang, 2006. "New Approaches to Ranking Economics Journals," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 5(1), pages 1-44, August.
    5. Sauer, Raymond D, 1988. "Estimates of the Returns to Quality and Coauthorship in Economic Academia," Journal of Political Economy, University of Chicago Press, vol. 96(4), pages 855-866, August.
    6. Stan J. Liebowitz, 2014. "Willful Blindness: The Inefficient Reward Structure In Academic Research," Economic Inquiry, Western Economic Association International, vol. 52(4), pages 1267-1283, October.
    7. Christiana E. Hilmer & Michael J. Hilmer, 2005. "How Do Journal Quality, Co-Authorship, and Author Order Affect Agricultural Economists' Salaries?," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 87(2), pages 509-523.
    8. David L. Anderson & Warren Smart & John Tressler, 2013. "Evaluating research -- peer review team assessment and journal based bibliographic measures: New Zealand PBRF research output scores in 2006," New Zealand Economic Papers, Taylor & Francis Journals, vol. 47(2), pages 140-157, August.
    9. Bernt Bratsberg & James F. Ragan & John T. Warren, 2010. "Does Raiding Explain The Negative Returns To Faculty Seniority?," Economic Inquiry, Western Economic Association International, vol. 48(3), pages 704-721, July.
    10. Christiana E. Hilmer & Michael J. Hilmer & Jayson L. Lusk, 2012. "A Comparison of Salary Structures Between Economics and Agricultural Economics Departments," Applied Economic Perspectives and Policy, Agricultural and Applied Economics Association, vol. 34(3), pages 489-514.
    11. John Gibson & David L. Anderson & John Tressler, 2014. "Which Journal Rankings Best Explain Academic Salaries? Evidence From The University Of California," Economic Inquiry, Western Economic Association International, vol. 52(4), pages 1322-1340, October.
    12. David I Stern, 2014. "High-Ranked Social Science Journal Articles Can Be Identified from Early Citation Information," PLOS ONE, Public Library of Science, vol. 9(11), pages 1-11, November.
    13. Vuong, Quang H, 1989. "Likelihood Ratio Tests for Model Selection and Non-nested Hypotheses," Econometrica, Econometric Society, vol. 57(2), pages 307-333, March.
    14. Glenn Ellison, 2002. "Evolving Standards for Academic Publishing: A q-r Theory," Journal of Political Economy, University of Chicago Press, vol. 110(5), pages 994-1034, October.
    15. Andrew J. Oswald, 2007. "An Examination of the Reliability of Prestigious Scholarly Journals: Evidence and Implications for Decision‐Makers," Economica, London School of Economics and Political Science, vol. 74(293), pages 21-31, February.
    16. Gibson, John, 2014. "Returns to articles versus pages in academic publishing: Do salary-setters show ‘article illusion’?," Economics Letters, Elsevier, vol. 125(3), pages 343-346.
    17. Bernt Bratsberg & James F. Ragan Jr. & John T. Warren, 2003. "Negative Returns to Seniority: New Evidence in Academic Markets," ILR Review, Cornell University, ILR School, vol. 56(2), pages 306-323, January.
    18. Ransom, Michael R, 1993. "Seniority and Monopsony in the Academic Labor Market," American Economic Review, American Economic Association, vol. 83(1), pages 221-233, March.
    19. Scott, Loren C & Mitias, Peter M, 1996. "Trends in Rankings of Economics Departments in the U.S.: An Update," Economic Inquiry, Western Economic Association International, vol. 34(2), pages 378-400, April.
    20. Pierre-Philippe Combes & Laurent Linnemer, 2010. "Inferring Missing Citations: A Quantitative Multi-Criteria Ranking of all Journals in Economics," Working Papers halshs-00520325, HAL.
    21. Matthew Spiegel, 2012. "Reviewing Less--Progressing More," Review of Financial Studies, Society for Financial Studies, vol. 25(5), pages 1331-1338.
    22. Joseph Macri & Dipendra Sinha, 2006. "Rankings Methodology for International Comparisons of Institutions and Individuals: an Application to Economics in Australia and New Zealand," Journal of Economic Surveys, Wiley Blackwell, vol. 20(1), pages 111-156, February.
    23. Laband, David N & Piette, Michael J, 1994. "The Relative Impacts of Economics Journals: 1970-1990," Journal of Economic Literature, American Economic Association, vol. 32(2), pages 640-666, June.
    24. Liebowitz, S J & Palmer, J P, 1984. "Assessing the Relative Impacts of Economic Journals," Journal of Economic Literature, American Economic Association, vol. 22(1), pages 77-88, March.
    25. John Gibson, 2000. "Research productivity in New Zealand university economics departments: Comment and update," New Zealand Economic Papers, Taylor & Francis Journals, vol. 34(1), pages 73-87.
    26. Daniel S. Hamermesh & Gerard A. Pfann, 2012. "Reputation And Earnings: The Roles Of Quality And Quantity In Academe," Economic Inquiry, Western Economic Association International, vol. 50(1), pages 1-16, January.
    27. repec:wly:soecon:v:82:2:y:2015:p:430-452 is not listed on IDEAS
    28. Stephan B. Bruns & David I. Stern, 2016. "Research assessment using early citation information," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(2), pages 917-935, August.
    29. Michael J. Hilmer & Christiana E. Hilmer, 2011. "Negative Returns to Seniority and Job Mobility across the Program Quality Distribution: Are Top Public PhD-Granting Programs Different?," American Economic Review, American Economic Association, vol. 101(3), pages 466-470, May.
    30. Moore, William J & Newman, Robert J & Turnbull, Geoffrey K, 1998. "Do Academic Salaries Decline with Seniority?," Journal of Labor Economics, University of Chicago Press, vol. 16(2), pages 352-366, April.
    31. Hicks, Diana, 2012. "Performance-based university research funding systems," Research Policy, Elsevier, vol. 41(2), pages 251-261.
    32. Tom Coupé, 2003. "Revealed Performances: Worldwide Rankings of Economists and Economics Departments, 1990-2000," Journal of the European Economic Association, MIT Press, vol. 1(6), pages 1309-1345, December.
    33. Moore, William J & Newman, Robert J & Turnbull, Geoffrey K, 2001. "Reputational Capital and Academic Pay," Economic Inquiry, Western Economic Association International, vol. 39(4), pages 663-671, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Frode Eika Sandnes, 2018. "Do Norwegian academics who publish more earn higher salaries?," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 263-281, April.
    2. María Victoria Anauati & Sebastian Galiani & Ramiro H. Gálvez, 2020. "Differences In Citation Patterns Across Journal Tiers: The Case Of Economics," Economic Inquiry, Western Economic Association International, vol. 58(3), pages 1217-1232, July.
    3. Lutz Bornmann & Klaus Wohlrabe, 2019. "Normalisation of citation impact in economics," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(2), pages 841-884, August.
    4. María Victoria Anauati & Sebastian Galiani & Ramiro H. Gálvez, 2018. "Differences in citation patterns across journal tiers in economics," Documentos de Trabajo 16701, The Latin American and Caribbean Economic Association (LACEA).
    5. Adam McCloskey & Pascal Michaillat, 2020. "Critical Values Robust to P-hacking," Papers 2005.04141, arXiv.org, revised Dec 2023.
    6. Yihui Lan & Kenneth W. Clements & Zong Ken Chai, 2023. "How Productive Are Economics and Finance PhDs?," Australian Economic Review, The University of Melbourne, Melbourne Institute of Applied Economic and Social Research, vol. 56(4), pages 442-461, December.
    7. Syed Hasan & Robert Breunig, 2021. "Article length and citation outcomes," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 7583-7608, September.
    8. Koffi, Marlene, 2021. "Innovative ideas and gender inequality," CLEF Working Paper Series 35, Canadian Labour Economics Forum (CLEF), University of Waterloo.
    9. Andrew Hussey & Sheena Murray & Wendy Stock, 2022. "Gender, coauthorship, and academic outcomes in economics," Economic Inquiry, Western Economic Association International, vol. 60(2), pages 465-484, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. David L. Anderson & John Tressler, 2017. "Researcher rank stability across alternative output measurement schemes in the context of a time limited research evaluation: the New Zealand case," Applied Economics, Taylor & Francis Journals, vol. 49(45), pages 4542-4553, September.
    2. John Gibson & David L. Anderson & John Tressler, 2014. "Which Journal Rankings Best Explain Academic Salaries? Evidence From The University Of California," Economic Inquiry, Western Economic Association International, vol. 52(4), pages 1322-1340, October.
    3. David L. Anderson & John Tressler, 2015. "Are Researcher Rankings Stable Across Alternative Output Measurement Schemes in the Context of a Time Limited Research Evaluation? The New Zealand Case," Working Papers in Economics 15/10, University of Waikato.
    4. David L. Anderson & John Tressler, 2013. "The New Zealand performance-based research fund and its impact on publication activity in economics," Research Evaluation, Oxford University Press, vol. 23(1), pages 1-11, September.
    5. Michael J. Hilmer & Michael R. Ransom & Christiana E. Hilmer, 2015. "Fame and the fortune of academic economists: How the market rewards influential research in economics," Southern Economic Journal, John Wiley & Sons, vol. 82(2), pages 430-452, October.
    6. David L. Anderson & John Tressler, 2011. "Ranking Economics Departments In Terms Of Residual Productivity: New Zealand Economics Departments, 2000–2006," Australian Economic Papers, Wiley Blackwell, vol. 50(4), pages 157-168, December.
    7. Daniel S. Hamermesh, 2018. "Citations in Economics: Measurement, Uses, and Impacts," Journal of Economic Literature, American Economic Association, vol. 56(1), pages 115-156, March.
    8. Damien Besancenot & Abdelghani Maddi, 2019. "Should citations be weighted to assess the influence of an academic article?," Economics Bulletin, AccessEcon, vol. 435(1), pages 435-445.
    9. Michael J. Hilmer & Christiana E. Hilmer, 2009. "Fishes, Ponds, And Productivity: Student‐Advisor Matching And Early Career Publishing Success For Economics Phds," Economic Inquiry, Western Economic Association International, vol. 47(2), pages 290-303, April.
    10. David L. Anderson & John Tressler, 2016. "Citation-Capture Rates for Economics Journals: Do they Differ from Other Disciplines and Does it Matter?," Economic Papers, The Economic Society of Australia, vol. 35(1), pages 73-85, March.
    11. João Ricardo Faria & Franklin G. Mixon, 2021. "The Marginal Impact of a Publication on Citations, and Its Effect on Academic Pay," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 8217-8226, September.
    12. David L. Anderson & John Tressler, 2011. "The Merits of Using Citations to Measure Research Output in Economics Departments: The New Zealand Case," Working Papers in Economics 11/11, University of Waikato.
    13. David Anderson & John Tressler, 2008. "Research output in New Zealand economics departments 2000-2006: A stock approach," New Zealand Economic Papers, Taylor & Francis Journals, vol. 42(2), pages 155-189.
    14. David L. Anderson & John Tressler, 2013. "The Relevance of the “h-” and “g-” Index to Economics in the Context of A Nation-Wide Research Evaluation Scheme: The New Zealand Case," Economic Papers, The Economic Society of Australia, vol. 32(1), pages 81-94, March.
    15. Syed Hasan & Robert Breunig, 2021. "Article length and citation outcomes," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 7583-7608, September.
    16. Koffi, Marlene, 2021. "Innovative ideas and gender inequality," CLEF Working Paper Series 35, Canadian Labour Economics Forum (CLEF), University of Waterloo.
    17. Gibson, John, 2014. "Returns to articles versus pages in academic publishing: Do salary-setters show ‘article illusion’?," Economics Letters, Elsevier, vol. 125(3), pages 343-346.
    18. Hilmer, Christiana E. & Hilmer, Michael J., 2020. "On the labor market for full-time non-tenure-track lecturers in economics," Economics of Education Review, Elsevier, vol. 78(C).
    19. Vasilios D. Kosteas, 2018. "Predicting long-run citation counts for articles in top economics journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(3), pages 1395-1412, June.
    20. David L. Anderson & John Tressler, 2010. "The Merits of Using Citation-Based Journal Weighting Schemes to Measure Research Performance in Economics: The Case of New Zealand," Working Papers in Economics 10/03, University of Waikato.

    More about this item

    JEL classification:

    • A14 - General Economics and Teaching - - General Economics - - - Sociology of Economics
    • J44 - Labor and Demographic Economics - - Particular Labor Markets - - - Professional Labor Markets and Occupations

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:ecinqu:v:55:y:2017:i:4:p:1945-1965. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/weaaaea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.