IDEAS home Printed from
   My bibliography  Save this article

How to avoid borrowed plumes in academia


  • Osterloh, Margit
  • Frey, Bruno S.


Publications in top journals today have a powerful influence on academic careers although there is much criticism of using journal rankings to evaluate individual articles. We ask why this practice of performance evaluation is still so influential. We suggest this is the case because a majority of authors benefit from the present system due to the extreme skewness of citation distributions. “Performance paradox” effects aggravate the problem. Three extant suggestions for reforming performance management are critically discussed. We advance a new proposal based on the insight that fundamental uncertainty is symptomatic for scholarly work. It suggests focal randomization using a rationally founded and well-orchestrated procedure.

Suggested Citation

  • Osterloh, Margit & Frey, Bruno S., 2020. "How to avoid borrowed plumes in academia," Research Policy, Elsevier, vol. 49(1).
  • Handle: RePEc:eee:respol:v:49:y:2020:i:1:s0048733319301519
    DOI: 10.1016/j.respol.2019.103831

    Download full text from publisher

    File URL:
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL:
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    1. Paul Nightingale & Alister Scott, 2007. "Peer review and the relevance gap: Ten suggestions for policy-makers," Science and Public Policy, Oxford University Press, vol. 34(8), pages 543-553, October.
    2. Frieder Michel Paulus & Lena Rademacher & Theo Alexander Jose Schäfer & Laura Müller-Pinzler & Sören Krach, 2015. "Journal Impact Factor Shapes Scientists’ Reward Signal in the Prospect of Publication," PLOS ONE, Public Library of Science, vol. 10(11), pages 1-15, November.
    3. Alexandru T. Balaban, 2012. "Positive and negative aspects of citation indices and journal impact factors," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 241-247, August.
    4. Mingers, John & Xu, Fang, 2010. "The drivers of citations in management science journals," European Journal of Operational Research, Elsevier, vol. 205(2), pages 422-430, September.
    5. David N. Laband & Robert D. Tollison, 2003. "Dry Holes in Economic Research," Kyklos, Wiley Blackwell, vol. 56(2), pages 161-173, May.
    6. V. Ginsburgh & Sheila Weyers, 2014. "Nominees, winners, and losers," Journal of Cultural Economics, Springer;The Association for Cultural Economics International, vol. 38(4), pages 291-313, November.
    7. Katja Rost & Thorsten Teichert & Alan Pilkington, 2017. "Social network analytics for advanced bibliometrics: referring to actor roles of management journals instead of journal rankings," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(3), pages 1631-1657, September.
    8. Wang, Jian & Veugelers, Reinhilde & Stephan, Paula, 2017. "Bias against novelty in science: A cautionary tale for users of bibliometric indicators," Research Policy, Elsevier, vol. 46(8), pages 1416-1436.
    9. James J. Heckman & Sidharth Moktan, 2020. "Publishing and Promotion in Economics: The Tyranny of the Top Five," Journal of Economic Literature, American Economic Association, vol. 58(2), pages 419-470, June.
    10. Bruno Frey, 2009. "Economists in the PITS?," International Review of Economics, Springer;Happiness Economics and Interpersonal Relations (HEIRS), vol. 56(4), pages 335-346, December.
    11. Bruno S. Frey & Katja Rost, 2010. "Do rankings reflect research quality?," Journal of Applied Economics, Universidad del CEMA, vol. 13, pages 1-38, May.
    12. Goodall, Amanda H., 2009. "Highly cited leaders and the performance of research universities," Research Policy, Elsevier, vol. 38(7), pages 1079-1092, September.
    13. Andrew J. Oswald, 2007. "An Examination of the Reliability of Prestigious Scholarly Journals: Evidence and Implications for Decision‐Makers," Economica, London School of Economics and Political Science, vol. 74(293), pages 21-31, February.
    14. Mårtensson, Pär & Fors, Uno & Wallin, Sven-Bertil & Zander, Udo & Nilsson, Gunnar H, 2016. "Evaluating research: A multidisciplinary approach to assessing research practice and quality," Research Policy, Elsevier, vol. 45(3), pages 593-603.
    15. Vincent Larivière & Yves Gingras, 2010. "The impact factor's Matthew Effect: A natural experiment in bibliometrics," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(2), pages 424-427, February.
    16. Stephan, Paula E., 2010. "The Economics of Science," Handbook of the Economics of Innovation, in: Bronwyn H. Hall & Nathan Rosenberg (ed.), Handbook of the Economics of Innovation, edition 1, volume 1, chapter 0, pages 217-273, Elsevier.
    17. Qiang Zhi & Tianguang Meng, 2016. "Funding allocation, inequality, and scientific research output: an empirical study based on the life science sector of Natural Science Foundation of China," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(2), pages 603-628, February.
    18. Vogel, Rick & Hattke, Fabian & Petersen, Jessica, 2017. "Journal rankings in management and business studies: What rules do we play by?," Research Policy, Elsevier, vol. 46(10), pages 1707-1722.
    19. Giovanni Dosi & Franco Malerba & Giovanni B. Ramello & Francesco Silva, 2006. "Information, appropriability, and the generation of innovative knowledge four decades after Arrow and Nelson: an introduction," Industrial and Corporate Change, Oxford University Press, vol. 15(6), pages 891-901, December.
    20. Henk F Moed, 2007. "The future of research evaluation rests with an intelligent combination of advanced metrics and transparent peer review," Science and Public Policy, Oxford University Press, vol. 34(8), pages 575-583, October.
    21. Peter Stolz, 1986. "Parteienwettbewerb, politisches Kartell und Tausch zwischen sozioökonomischen Gruppen," Swiss Journal of Economics and Statistics (SJES), Swiss Society of Economics and Statistics (SSES), vol. 122(IV), pages 657-675, December.
    22. Éric Archambault & Vincent Larivière, 2009. "History of the journal impact factor: Contingencies and consequences," Scientometrics, Springer;Akadémiai Kiadó, vol. 79(3), pages 635-649, June.
    23. Stuart Macdonald & Jacqueline Kam, 2007. "Ring a Ring o’ Roses: Quality Journals and Gamesmanship in Management Studies," Journal of Management Studies, Wiley Blackwell, vol. 44(4), pages 640-655, June.
    24. Wallace, Matthew L. & Larivière, Vincent & Gingras, Yves, 2009. "Modeling a century of citation distributions," Journal of Informetrics, Elsevier, vol. 3(4), pages 296-303.
    25. Richard R. Nelson, 2006. "The Market Economy and the Scientific Commons," Chapters, in: Birgitte Andersen (ed.), Intellectual Property Rights, chapter 1, Edward Elgar Publishing.
    26. Joshua S. Gans & George B. Shepherd, 1994. "How Are the Mighty Fallen: Rejected Classic Articles by Leading Economists," Journal of Economic Perspectives, American Economic Association, vol. 8(1), pages 165-179, Winter.
    27. Richard R. Nelson, 1959. "The Simple Economics of Basic Scientific Research," Journal of Political Economy, University of Chicago Press, vol. 67, pages 297-297.
    28. Elise S Brezis, 2007. "Focal randomisation: An optimal mechanism for the evaluation of R&D projects," Science and Public Policy, Oxford University Press, vol. 34(10), pages 691-698, December.
    29. Wilde, Christian & Krahnen, Jan Pieter & Ockenfels, Peter, 2014. "Measuring Ambiguity Aversion: A Systematic Experimental Approach," VfS Annual Conference 2014 (Hamburg): Evidence-based Economic Policy 100557, Verein für Socialpolitik / German Economic Association.
    30. Mats Alvesson & Jörgen Sandberg, 2013. "Has Management Studies Lost Its Way? Ideas for More Imaginative and Innovative Research," Journal of Management Studies, Wiley Blackwell, vol. 50(1), pages 128-152, January.
    31. Daniel S. Hamermesh, 2018. "Citations in Economics: Measurement, Uses, and Impacts," Journal of Economic Literature, American Economic Association, vol. 56(1), pages 115-156, March.
    32. David N. Laband, 2013. "On the Use and Abuse of Economics Journal Rankings," Economic Journal, Royal Economic Society, vol. 0, pages 223-254, August.
    33. Vincent Larivière & Yves Gingras, 2010. "The impact factor's Matthew Effect: A natural experiment in bibliometrics," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 61(2), pages 424-427, February.
    34. Frey, Bruno S, 2003. "Publishing as Prostitution?--Choosing between One's Own Ideas and Academic Success," Public Choice, Springer, vol. 116(1-2), pages 205-223, July.
    35. Partha, Dasgupta & David, Paul A., 1994. "Toward a new economics of science," Research Policy, Elsevier, vol. 23(5), pages 487-521, September.
    36. William H. Starbuck, 2005. "How Much Better Are the Most-Prestigious Journals? The Statistics of Academic Publication," Organization Science, INFORMS, vol. 16(2), pages 180-200, April.
    37. Juan Miguel Campanario, 2009. "Rejecting and resisting Nobel class discoveries: accounts by Nobel Laureates," Scientometrics, Springer;Akadémiai Kiadó, vol. 81(2), pages 549-565, November.
    38. J Mingers, 2008. "Exploring the dynamics of journal citations: Modelling with s-curves," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 59(8), pages 1013-1025, August.
    39. Michelle Gittelman & Bruce Kogut, 2003. "Does Good Science Lead to Valuable Knowledge? Biotechnology Firms and the Evolutionary Logic of Citation Patterns," Management Science, INFORMS, vol. 49(4), pages 366-382, April.
    40. Macdonald, Stuart & Kam, Jacqueline, 2010. "Counting footnotes: Citability in management studies," Scandinavian Journal of Management, Elsevier, vol. 26(2), pages 189-203, June.
    41. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    Full references (including those not matched with items on IDEAS)


    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.

    Cited by:

    1. van Dalen, Hendrik Peter, 2021. "How the publish-or-perish principle divides a science: The case of economists," Other publications TiSEM a6a5a855-bb5a-4d52-a841-3, Tilburg University, School of Economics and Management.
    2. Amanda Goodall & Margit Osterloh & Mandy Fong, 2020. "Women Shy Away From Competition – How To Overcome It," CREMA Working Paper Series 2020-21, Center for Research in Economics, Management and the Arts (CREMA).
    3. Peter Andre & Armin Falk, 2021. "What’s Worth Knowing? Economists’ Opinions about Economics," ECONtribute Discussion Papers Series 102, University of Bonn and University of Cologne, Germany.
    4. Drivas, Kyriakos & Kremmydas, Dimitris, 2020. "The Matthew effect of a journal's ranking," Research Policy, Elsevier, vol. 49(4).
    5. Tran, Trung & Linh, Hoang Khanh & La, Viet-Phuong & Ho, Manh-Toan & Vuong, Quan-Hoang, 2020. "Scrambling for higher metrics in the Journal Impact Factor bubble period: a real-world problem in science management and its implications," OSF Preprints dmsp9, Center for Open Science.
    6. Follert, Florian & Naumann, Chantal & Thieme, Lutz, 2020. "Between scientific publication and public perception: Some economic remarks on the allocation of time in science," Working Papers of the European Institute for Socioeconomics 34, European Institute for Socioeconomics (EIS), Saarbrücken.
    7. Marta Kuc-Czarnecka & Magdalena Olczyk, 2020. "How ethics combine with big data: a bibliometric analysis," Palgrave Communications, Palgrave Macmillan, vol. 7(1), pages 1-9, December.
    8. Gabriel-Alexandru Vîiu & Mihai Păunescu, 2021. "The citation impact of articles from which authors gained monetary rewards based on journal metrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(6), pages 4941-4974, June.
    9. Hendrik P. Dalen, 2021. "How the publish-or-perish principle divides a science: the case of economists," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1675-1694, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Drivas, Kyriakos & Kremmydas, Dimitris, 2020. "The Matthew effect of a journal's ranking," Research Policy, Elsevier, vol. 49(4).
    2. Margit Osterloh & Bruno S. Frey, 2009. "Research Governance in Academia: Are there Alternatives to Academic Rankings?," CESifo Working Paper Series 2797, CESifo.
    3. Margit Osterloh & Bruno S. Frey, 2010. "Academic rankings and research governance," IEW - Working Papers 482, Institute for Empirical Research in Economics - University of Zurich.
    4. Katja Rost & Bruno S. Frey, 2011. "Quantitative and Qualitative Rankings of Scholars," Schmalenbach Business Review (sbr), LMU Munich School of Management, vol. 63(1), pages 63-91, January.
    5. Dell'Anno, Roberto & Caferra, Rocco & Morone, Andrea, 2020. "A “Trojan Horse” in the peer-review process of fee-charging economic journals," Journal of Informetrics, Elsevier, vol. 14(3).
    6. Rodríguez Sánchez, Isabel & Makkonen, Teemu & Williams, Allan M., 2019. "Peer review assessment of originality in tourism journals: critical perspective of key gatekeepers," Annals of Tourism Research, Elsevier, vol. 77(C), pages 1-11.
    7. Gans, Joshua S. & Murray, Fiona E. & Stern, Scott, 2017. "Contracting over the disclosure of scientific knowledge: Intellectual property and academic publication," Research Policy, Elsevier, vol. 46(4), pages 820-835.
    8. Matthias Aistleitner & Jakob Kapeller & Stefan Steinerberger, 2018. "Citation Patterns in Economics and Beyond," Working Papers Series 85, Institute for New Economic Thinking.
    9. Veugelers, Reinhilde & Wang, Jian, 2019. "Scientific novelty and technological impact," Research Policy, Elsevier, vol. 48(6), pages 1362-1372.
    10. Joseph Gerald Hirschberg & Jeanette Ngaire Lye, 2020. "Grading Journals In Economics: The Abcs Of The Abdc," Journal of Economic Surveys, Wiley Blackwell, vol. 34(4), pages 876-921, September.
    11. van Dalen, Hendrik Peter, 2020. "How the Publish-or-Perish Principle Divides a Science : The Case of Academic Economists," Other publications TiSEM 6fbb6b92-0e06-4271-b6e7-3, Tilburg University, School of Economics and Management.
    12. van Dalen, Hendrik Peter, 2021. "How the publish-or-perish principle divides a science: The case of economists," Other publications TiSEM a6a5a855-bb5a-4d52-a841-3, Tilburg University, School of Economics and Management.
    13. J. Atsu Amegashie, 2020. "Citations And Incentives In Academic Contests," Economic Inquiry, Western Economic Association International, vol. 58(3), pages 1233-1244, July.
    14. Simeth, Markus & Raffo, Julio D., 2013. "What makes companies pursue an Open Science strategy?," Research Policy, Elsevier, vol. 42(9), pages 1531-1543.
    15. Mudambi, Ram & Swift, Tim, 2009. "Professional guilds, tension and knowledge management," Research Policy, Elsevier, vol. 38(5), pages 736-745, June.
    16. Simeth, Markus & Lhuillery, Stephane, 2015. "How do firms develop capabilities for scientific disclosure?," Research Policy, Elsevier, vol. 44(7), pages 1283-1295.
    17. Damien Besancenot & Kim Huynh & Joao Faria, 2012. "Search and research: the influence of editorial boards on journals’ quality," Theory and Decision, Springer, vol. 73(4), pages 687-702, October.
    18. Domingo Docampo & Vicente Safón, 2021. "Journal ratings: a paper affiliation methodology," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 8063-8090, September.
    19. Schaeffer, Véronique, 2019. "The use of material transfer agreements in academia: A threat to open science or a cooperation tool?," Research Policy, Elsevier, vol. 48(9), pages 1-1.
    20. Jung, Hyun Ju & Lee, Jeongsik “Jay”, 2014. "The impacts of science and technology policy interventions on university research: Evidence from the U.S. National Nanotechnology Initiative," Research Policy, Elsevier, vol. 43(1), pages 74-91.


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:respol:v:49:y:2020:i:1:s0048733319301519. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.