IDEAS home Printed from https://ideas.repec.org/a/sae/evarev/v39y2015i1p102-129.html
   My bibliography  Save this article

Ranking Games

Author

Listed:
  • Margit Osterloh
  • Bruno S. Frey

Abstract

Background: Research rankings based on bibliometrics today dominate governance in academia and determine careers in universities. Method: Analytical approach to capture the incentives by users of rankings and by suppliers of rankings, both on an individual and an aggregate level. Result: Rankings may produce unintended negative side effects. In particular, rankings substitute the “taste for science†by a “taste for publication.†We show that the usefulness of rankings rests on several important assumptions challenged by recent research. Conclusion: We suggest as alternatives careful socialization and selection of scholars, supplemented by periodic self-evaluations and awards. The aim is to encourage controversial discourses in order to contribute meaningful to the advancement of science.

Suggested Citation

  • Margit Osterloh & Bruno S. Frey, 2015. "Ranking Games," Evaluation Review, , vol. 39(1), pages 102-129, February.
    • Handle: RePEc:sae:evarev:v:39:y:2015:i:1:p:102-129
    DOI: 10.1177/0193841X14524957
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0193841X14524957
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0193841X14524957?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Sönke Albers, 2009. "Misleading Rankings of Research in Business," German Economic Review, Verein für Socialpolitik, vol. 10(3), pages 352-363, August.
    2. Butler, Linda, 2003. "Explaining Australia's increased share of ISI publications--the effects of a funding formula based on publication counts," Research Policy, Elsevier, vol. 32(1), pages 143-155, January.
    3. Matthias Benz & Bruno S. Frey, "undated". "Corporate Governance: What can we Learn from Public Governance?," IEW - Working Papers 166, Institute for Empirical Research in Economics - University of Zurich.
    4. Ernst Fehr & Klaus M. Schmidt, 2004. "Fairness and Incentives in a Multi‐task Principal–Agent Model," Scandinavian Journal of Economics, Wiley Blackwell, vol. 106(3), pages 453-474, October.
    5. Abernethy, Margaret A. & Brownell, Peter, 1997. "Management control systems in research and development organizations: The role of accounting, behavior and personnel controls," Accounting, Organizations and Society, Elsevier, vol. 22(3-4), pages 233-248.
    6. Linda Butler, 2007. "Assessing university research: A plea for a balanced approach," Science and Public Policy, Oxford University Press, vol. 34(8), pages 565-574, October.
    7. Sönke Albers, 2009. "Misleading Rankings of Research in Business," German Economic Review, Verein für Socialpolitik, vol. 10(3), pages 352-363, August.
    8. Universities-National Bureau Committee for Economic Research & Committee on Economic Growth of the Social Science Research Council, 1962. "The Rate and Direction of Inventive Activity: Economic and Social Factors," NBER Books, National Bureau of Economic Research, Inc, number univ62-1, March.
    9. Sendil K. Ethiraj & Daniel Levinthal, 2009. "Hoping for A to Z While Rewarding Only A: Complex Organizations and Multiple Goals," Organization Science, INFORMS, vol. 20(1), pages 4-21, February.
    10. Richard Nelson, 1962. "Introduction to "The Rate and Direction of Inventive Activity: Economic and Social Factors"," NBER Chapters, in: The Rate and Direction of Inventive Activity: Economic and Social Factors, pages 1-16, National Bureau of Economic Research, Inc.
    11. Michael Kosfeld & Armin Falk, 2006. "The Hidden Costs of Control," American Economic Review, American Economic Association, vol. 96(5), pages 1611-1630, December.
    12. Claire Donovan, 2007. "The qualitative future of research evaluation," Science and Public Policy, Oxford University Press, vol. 34(8), pages 585-597, October.
    13. Kathleen M. Eisenhardt, 1985. "Control: Organizational and Economic Approaches," Management Science, INFORMS, vol. 31(2), pages 134-149, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Blog mentions

    As found by EconAcademics.org, the blog aggregator for Economics research:
    1. Bruno Frey's academic utopia
      by Economic Logician in Economic Logic on 2011-09-27 19:38:00

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. repec:esx:essedp:757 is not listed on IDEAS
    2. Régibeau, P & Rockett, K, 2014. "A Tale of Two Metrics: Research Assessment vs Recognised Excellence," Economics Discussion Papers 14461, University of Essex, Department of Economics.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Margit Osterloh & Bruno S. Frey, 2010. "Academic rankings and research governance," IEW - Working Papers 482, Institute for Empirical Research in Economics - University of Zurich.
    2. Margit Osterloh & Bruno S. Frey, 2009. "Research Governance in Academia: Are there Alternatives to Academic Rankings?," CREMA Working Paper Series 2009-17, Center for Research in Economics, Management and the Arts (CREMA).
    3. Margit Osterloh & Bruno S. Frey, 2013. "Motivation governance," Chapters, in: Anna Grandori (ed.), Handbook of Economic Organization, chapter 2, Edward Elgar Publishing.
    4. Bruno S. Frey & Margit Osterloh, 2006. "Evaluations: Hidden Costs, Questionable Benefits, and Superior Alternatives," CREMA Working Paper Series 2006-23, Center for Research in Economics, Management and the Arts (CREMA), revised Oct 2006.
    5. Bruno S. Frey & Fabian Homberg & Margit Osterloh, 2013. "Organizational Control Systems and Pay-for-Performance in the Public Service," CREMA Working Paper Series 2013-11, Center for Research in Economics, Management and the Arts (CREMA).
    6. Pierre Koning & J. Vyrastekova & S. Onderstal, 2006. "Team incentives in public organisations; an experimental study," CPB Discussion Paper 60, CPB Netherlands Bureau for Economic Policy Analysis.
    7. Müller, Harry, 2012. "Die Zitationshäufigkeit als Qualitätsindikator im Rahmen der Forschungsleistungsmessung," Discussion Papers of the Institute for Organisational Economics 1/2012, University of Münster, Institute for Organisational Economics.
    8. Christian-Albrechts-Universität zu Kiel, Institut für Betriebswirtschaftslehre (Ed.), 2010. "Jahresbericht 2009," Manuskripte aus den Instituten für Betriebswirtschaftslehre der Universität Kiel 650, Christian-Albrechts-Universität zu Kiel, Institut für Betriebswirtschaftslehre.
    9. Emil Inauen & Katja Rost & Margit Osterloh & Bruno S. Frey, 2010. "Back to the Future –A Monastic Perspective on Corporate Governance," management revue - Socio-Economic Studies, Nomos Verlagsgesellschaft mbH & Co. KG, vol. 21(1), pages 38-59.
    10. David Paul A., 2008. "The Historical Origins of 'Open Science': An Essay on Patronage, Reputation and Common Agency Contracting in the Scientific Revolution," Capitalism and Society, De Gruyter, vol. 3(2), pages 1-106, October.
    11. Justus Haucap & Johannes Muck, 2015. "What drives the relevance and reputation of economics journals? An update from a survey among economists," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(3), pages 849-877, June.
    12. Laura Magazzini & Fabio Pammolli & Massimo Riccaboni & Maria Alessandra Rossi, 2009. "Patent disclosure and R&D competition in pharmaceuticals," Economics of Innovation and New Technology, Taylor & Francis Journals, vol. 18(5), pages 467-486.
    13. Williams, Christopher & van Triest, Sander, 2009. "The impact of corporate and national cultures on decentralization in multinational corporations," International Business Review, Elsevier, vol. 18(2), pages 156-167, April.
    14. Anja, Breitwieser & Neil, Foster, 2012. "Intellectual property rights, innovation and technology transfer: a survey," MPRA Paper 36094, University Library of Munich, Germany.
    15. Hanna Hottenrott & Bettina Peters, 2012. "Innovative Capability and Financing Constraints for Innovation: More Money, More Innovation?," The Review of Economics and Statistics, MIT Press, vol. 94(4), pages 1126-1142, November.
    16. Bruno S. Frey & Simon Luechinger & Alois Stutzer, 2007. "Calculating Tragedy: Assessing The Costs Of Terrorism," Journal of Economic Surveys, Wiley Blackwell, vol. 21(1), pages 1-24, February.
    17. Mario Coccia, 2006. "Classifications of innovations: Survey and future directions," CERIS Working Paper 200602, CNR-IRCrES Research Institute on Sustainable Economic Growth - Torino (TO) ITALY - former Institute for Economic Research on Firms and Growth - Moncalieri (TO) ITALY.
    18. Sebastian Goebel & Barbara Weißenberger, 2016. "The Dark Side of Tight Financial Control: Causes and Remedies of Dysfunctional Employee Behaviors," Schmalenbach Business Review, Springer;Schmalenbach-Gesellschaft, vol. 17(1), pages 69-101, April.
    19. Fagerberg, Jan & Fosaas, Morten & Sapprasert, Koson, 2012. "Innovation: Exploring the knowledge base," Research Policy, Elsevier, vol. 41(7), pages 1132-1153.
    20. Greve, Jan & Ax, Christian & Bedford, David S. & Bednarek, Piotr & Brühl, Rolf & Dergård, Johan & Ditillo, Angelo & Dossi, Andrea & Gosselin, Maurice & Hoozée, Sophie & Israelsen, Poul & Janschek, Ott, 2017. "The impact of society on management control systems," Scandinavian Journal of Management, Elsevier, vol. 33(4), pages 253-266.

    More about this item

    Keywords

    academic governance; rankings; motivation; selection; socialization;
    All these keywords.

    JEL classification:

    • A10 - General Economics and Teaching - - General Economics - - - General
    • D02 - Microeconomics - - General - - - Institutions: Design, Formation, Operations, and Impact
    • H83 - Public Economics - - Miscellaneous Issues - - - Public Administration
    • L23 - Industrial Organization - - Firm Objectives, Organization, and Behavior - - - Organization of Production
    • M50 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - Personnel Economics - - - General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:evarev:v:39:y:2015:i:1:p:102-129. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.