IDEAS home Printed from https://ideas.repec.org/a/eee/joepsy/v66y2018icp33-44.html
   My bibliography  Save this article

Lower-rated publications do lower academics’ judgments of publication lists: Evidence from a survey experiment of economists

Author

Listed:
  • Powdthavee, Nattavudh
  • Riyanto, Yohanes E.
  • Knetsch, Jack L.

Abstract

Publications in leading journals are widely known to have a positive impact on economists’ judgments of the value of authors’ contributions and professional reputations. While conjectures that publications in lower-rated journals likely have a negative impact on such judgments are common, there have been virtually no direct tests of their validity. Our intent is to provide results from such a test, one that involved asking economists from 44 universities throughout the world to rate either a publication list with only higher-rated journals or a list with all of these but with additional publications in lower-rated journals. Our primary finding was that, holding other things constant, adding publications in lower-rated journals to what is typically considered a good publication record does have a significant negative impact on economists’ judgments of the value of the author’s contribution. Most implications of this bias suggest negative impacts on social welfare.

Suggested Citation

  • Powdthavee, Nattavudh & Riyanto, Yohanes E. & Knetsch, Jack L., 2018. "Lower-rated publications do lower academics’ judgments of publication lists: Evidence from a survey experiment of economists," Journal of Economic Psychology, Elsevier, vol. 66(C), pages 33-44.
  • Handle: RePEc:eee:joepsy:v:66:y:2018:i:c:p:33-44
    DOI: 10.1016/j.joep.2018.04.003
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S016748701830059X
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.joep.2018.04.003?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. John P. Conley & Mario J. Crucini & Robert A. Driskill & Ali Sina Önder, 2013. "The Effects Of Publication Lags On Life-Cycle Research Productivity In Economics," Economic Inquiry, Western Economic Association International, vol. 51(2), pages 1251-1276, April.
    2. Wijeyakulasuriya D. A. & Wickremasinghe W. N., 2015. "Measuring Extreme Market Risk: The Sri Lankan Context," Asia-Pacific Journal of Risk and Insurance, De Gruyter, vol. 9(2), pages 185-201, July.
    3. Richard Dusansky & Clayton J. Vernon, 1998. "Rankings of U.S. Economics Departments," Journal of Economic Perspectives, American Economic Association, vol. 12(1), pages 157-170, Winter.
    4. Denis Bouyssou & Thierry Marchant, 2011. "Ranking scientists and departments in a consistent manner," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(9), pages 1761-1769, September.
    5. Kyle Jurado & Sydney C. Ludvigson & Serena Ng, 2015. "Measuring Uncertainty," American Economic Review, American Economic Association, vol. 105(3), pages 1177-1216, March.
    6. Gaut, Fred, 2015. "Measuring innovation in all sectors of the economy," MERIT Working Papers 2015-038, United Nations University - Maastricht Economic and Social Research Institute on Innovation and Technology (MERIT).
    7. David Card & Stefano DellaVigna, 2013. "Nine Facts about Top Journals in Economics," Journal of Economic Literature, American Economic Association, vol. 51(1), pages 144-161, March.
    8. Kalaitzidakis, P. & Mamuneas, T.P. & Stengos, T., 2003. "Rankings of Academic Journals and Institutions," Working Papers 2003-8, University of Guelph, Department of Economics and Finance.
    9. Steve Swidler & Elizabeth Goldreyer, 1998. "The Value of a Finance Journal Publication," Journal of Finance, American Finance Association, vol. 53(1), pages 351-363, February.
    10. ., 2015. "Legal measures for finalizing bargains," Chapters, in: Joint Venture Strategies, chapter 9, pages 132-143, Edward Elgar Publishing.
    11. Combes, Pierre-Philippe & Linnemer, Laurent & Visser, Michael, 2008. "Publish or peer-rich? The role of skills and networks in hiring economics professors," Labour Economics, Elsevier, vol. 15(3), pages 423-441, June.
    12. Kahn, Shulamit, 1993. "Gender Differences in Academic Career Paths of Economists," American Economic Review, American Economic Association, vol. 83(2), pages 52-56, May.
    13. Denis Bouyssou & Thierry Marchant, 2011. "Ranking scientists and departments in a consistent manner," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(9), pages 1761-1769, September.
    14. J. C. R. Alcantud & R. Andrés Calle & J. M. Cascón, 2015. "Pairwise Dichotomous Cohesiveness Measures," Group Decision and Negotiation, Springer, vol. 24(5), pages 833-854, September.
    15. Němec Daniel, 2015. "Measuring Inefficiency of the Czech Labour Market," Review of Economic Perspectives, Sciendo, vol. 15(2), pages 197-220, June.
    16. Pantelis Kalaitzidakis & Theofanis P. Mamuneas & Thanasis Stengos, 2003. "Rankings of Academic Journals and Institutions in Economics," Journal of the European Economic Association, MIT Press, vol. 1(6), pages 1346-1366, December.
    17. OLIVIER CADOT & Lili Yan Ing, 2015. "Non-Tariff Measures: Not All that Bad," Working Papers PB-2015-02, Economic Research Institute for ASEAN and East Asia (ERIA).
    18. Paul W. Grimes & Charles A. Register, 1997. "Career Publications and Academic Job Rank: Evidence from the Class of 1968," The Journal of Economic Education, Taylor & Francis Journals, vol. 28(1), pages 82-92, March.
    19. A. Nifo & G. Vecchione, 2015. "Measuring Institutional Quality in Italy," Rivista economica del Mezzogiorno, Società editrice il Mulino, issue 1-2, pages 157-182.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Syed Hasan & Robert Breunig, 2021. "Article length and citation outcomes," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 7583-7608, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Nattavudh Powdthavee & Yohanes E. Riyanto & Jack L. Knetsch, 2017. "Impact of Lower Rated Journals on Economists’ Judgments of Publication Lists: Evidence from a Survey Experiment," PIER Discussion Papers 63, Puey Ungphakorn Institute for Economic Research.
    2. Powdthavee, Nattavudh & Riyanto, Yohanes E. & Knetsch, Jack L., 2017. "Impact of Lower Rated Journals on Economists' Judgments of Publication Lists: Evidence from a Survey Experiment," IZA Discussion Papers 10752, Institute of Labor Economics (IZA).
    3. James J. Heckman & Sidharth Moktan, 2020. "Publishing and Promotion in Economics: The Tyranny of the Top Five," Journal of Economic Literature, American Economic Association, vol. 58(2), pages 419-470, June.
    4. José Alberto Molina & Alfredo Ferrer & David Iñiguez & Alejandro Rivero & Gonzalo Ruiz & Alfonso Tarancón, 2020. "Network analysis to measure academic performance in economics," Empirical Economics, Springer, vol. 58(3), pages 995-1018, March.
    5. Irion, Kristina & Helberger, Natali, 2017. "Smart TV and the online media sector: User privacy in view of changing market realities," Telecommunications Policy, Elsevier, vol. 41(3), pages 170-184.
    6. Önder Ali Sina & Yilmazkuday Hakan, 2020. "Thirty-Five Years of Peer-Reviewed Publishing by North American Economics PhDs: Quantity, Quality, and Beyond," Open Economics, De Gruyter, vol. 3(1), pages 70-85, January.
    7. Jang C. Jin, 2009. "Publications in mathematical economics and econometrics: ranking of Asian universities and an application of Zipf's law," Asian-Pacific Economic Literature, Asia Pacific School of Economics and Government, The Australian National University, vol. 23(2), pages 116-122, November.
    8. James B. Davies & Martin G. Kocher & Matthias Sutter, 2008. "Economics research in Canada: a long-run assessment of journal publications," Canadian Journal of Economics, Canadian Economics Association, vol. 41(1), pages 22-45, February.
    9. Lea†Rachel Kosnik, 2018. "A Survey Of Jel Codes: What Do They Mean And Are They Used Consistently?," Journal of Economic Surveys, Wiley Blackwell, vol. 32(1), pages 249-272, February.
    10. Baltagi, Badi H., 2007. "Worldwide Econometrics Rankings: 1989–2005," Econometric Theory, Cambridge University Press, vol. 23(5), pages 952-1012, October.
    11. Tilak Mukhopadhyay & Subrata Sarar, 2010. "Rankings of Economics Journals and Departments in India," Indira Gandhi Institute of Development Research, Mumbai Working Papers 2010-021, Indira Gandhi Institute of Development Research, Mumbai, India.
    12. Joseph Macri & Dipendra Sinha, 2006. "Rankings Methodology for International Comparisons of Institutions and Individuals: an Application to Economics in Australia and New Zealand," Journal of Economic Surveys, Wiley Blackwell, vol. 20(1), pages 111-156, February.
    13. Püttmann, Vitus & Thomsen, Stephan L. & Trunzer, Johannes, 2020. "Zur Relevanz von Ausstattungsunterschieden für Forschungsleistungsvergleiche: Ein Diskussionsbeitrag für die Wirtschaftswissenschaften in Deutschland," Hannover Economic Papers (HEP) dp-679, Leibniz Universität Hannover, Wirtschaftswissenschaftliche Fakultät, revised Mar 2021.
    14. J. Atsu Amegashie, 2020. "Citations And Incentives In Academic Contests," Economic Inquiry, Western Economic Association International, vol. 58(3), pages 1233-1244, July.
    15. William C. Horrace & Christopher F. Parmeter, 2017. "Accounting for Multiplicity in Inference on Economics Journal Rankings," Southern Economic Journal, John Wiley & Sons, vol. 84(1), pages 337-347, July.
    16. Seiler, Christian & Wohlrabe, Klaus, 2012. "Ranking economists on the basis of many indicators: An alternative approach using RePEc data," Journal of Informetrics, Elsevier, vol. 6(3), pages 389-402.
    17. J. A. García & Rosa Rodriguez-Sánchez & J. Fdez-Valdivia, 2012. "A comparison of top economics departments in the US and EU on the basis of the multidimensional prestige of influential articles in 2010," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(3), pages 681-698, December.
    18. Chagas , André Luis Squarize, 2017. "Publish or Perish: um ranking de revistas da subárea de Economia Regional e Urbana para os pesquisadores brasileiros," Revista Brasileira de Estudos Regionais e Urbanos, Associação Brasileira de Estudos Regionais e Urbanos (ABER), vol. 11(4), pages 515-536.
    19. Raúl Ramos & Vicente Royuela & Jordi Suriñach, 2007. "An analysis of the determinants in Economics and Business publications by Spanish universities between 1994 and 2004," Scientometrics, Springer;Akadémiai Kiadó, vol. 71(1), pages 117-144, April.

    More about this item

    Keywords

    Lower ranked journals; Publication; Judgment bias; Less-is-better effect;
    All these keywords.

    JEL classification:

    • A1 - General Economics and Teaching - - General Economics
    • M5 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - Personnel Economics

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:joepsy:v:66:y:2018:i:c:p:33-44. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://www.elsevier.com/locate/joep .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joep .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.