How Should Research Performance Be Measured? A Study of Swedish Economists
AbstractBillions of euros are allocated every year to university research. Increased specialisation and international integration of research and researchers has sharply raised the need for comparisons of performance across fields, institutions and individual researchers. However, there is still no consensus regarding how such rankings should be conducted and what output measures are appropriate to use. We rank all full professors in a particular discipline, economics, in one European nation using seven established, and some of them commonly used, measures of research performance. Our examination shows both that the rank order can vary greatly across measures, and that depending on the measure used the distribution of total research out-put is valued very differently. The renowned KMS measure in economics stands out among the measures analysed here. It exhibits the weakest correlation with the others used in our study. We conclude by giving advice to funding councils and others assessing research quality on how to think about the use of both quantitative and qualitative measures of performance.
Download InfoIf you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
Bibliographic InfoPaper provided by Stockholm School of Economics in its series Working Paper Series in Economics and Finance with number 693.
Length: 29 pages
Date of creation: 04 Mar 2008
Date of revision:
Publication status: Published as Henrekson, Magnus and Daniel Waldenström, 'How Should Research Performance Be Measured? A Study of Swedish Economists' in The Manchester School, 2011, pages 1139-1156.
Contact details of provider:
Postal: The Economic Research Institute, Stockholm School of Economics, P.O. Box 6501, 113 83 Stockholm, Sweden
Phone: +46-(0)8-736 90 00
Fax: +46-(0)8-31 01 57
Web page: http://www.hhs.se/
More information through EDIRC
Impact of research; Ranking; Research productivity; Bibliometrics; Impact Factor;
Other versions of this item:
- Magnus Henrekson & Daniel Waldenström, 2011. "How Should Research Performance Be Measured? A Study Of Swedish Economists," Manchester School, University of Manchester, vol. 79(6), pages 1139-1156, December.
- Henrekson, Magnus & Waldenström, Daniel, 2007. "How Should Research Performance be Measured? A Study of Swedish Economists," Working Paper Series 712, Research Institute of Industrial Economics, revised 08 Nov 2007.
- A11 - General Economics and Teaching - - General Economics - - - Role of Economics; Role of Economists
- A14 - General Economics and Teaching - - General Economics - - - Sociology of Economics
- B41 - Schools of Economic Thought and Methodology - - Economic Methodology - - - Economic Methodology
This paper has been announced in the following NEP Reports:
- NEP-ALL-2008-03-08 (All new papers)
- NEP-EDU-2008-03-08 (Education)
- NEP-EFF-2008-03-08 (Efficiency & Productivity)
- NEP-HRM-2008-03-08 (Human Capital & Human Resource Management)
- NEP-IPR-2008-03-08 (Intellectual Property Rights)
- NEP-SOG-2008-03-08 (Sociology of Economics)
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Pantelis Kalaitzidakis & Theofanis P. Mamuneas & Thanasis Stengos, 2003.
"Rankings of Academic Journals and Institutions in Economics,"
Journal of the European Economic Association,
MIT Press, vol. 1(6), pages 1346-1366, December.
- Pantelis Kalaitzidakis & Theofanis P Mamuneas & Thanasis Stengos, 2001. "Rankings of Academic Journals and Institutions in Economics," Discussion Papers in Economics 01/8, Department of Economics, University of Leicester.
- BAUWENS, Luc & KIRMAN, Alan & LUBRANO, Michel & PROTOPOPESCU, Camelia, .
"Ranking economics departments in Europe: a statistical approach,"
CORE Discussion Papers RP
-1694, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
- Michel Lubrano & Luc Bauwens & Alan Kirman & Camelia Protopopescu, 2003. "Ranking Economics Departments in Europe: A Statistical Approach," Journal of the European Economic Association, MIT Press, vol. 1(6), pages 1367-1401, December.
- BAUWENS, Luc & KIRMAN, Alan & LUBRANO, Michel & PROTOPOPESCU, Camelia, 2003. "Ranking economics departments in Europe: a statistical approach," CORE Discussion Papers 2003050, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
- Richard Dusansky & Clayton J. Vernon, 1998. "Rankings of U.S. Economics Departments," Journal of Economic Perspectives, American Economic Association, vol. 12(1), pages 157-170, Winter.
- David N. Laband & Robert D. Tollison, 2003. "Dry Holes in Economic Research," Kyklos, Wiley Blackwell, vol. 56(2), pages 161-173, 05.
RePEc Biblio mentionsAs found on the RePEc Biblio, the curated bibliography for Economics:
- > Economics Profession > Ranking in Economics
- > Economics Profession > Ranking in Economics > Ranking Methodology
- > Economics Profession > Ranking in Economics > Ranking Economists
- Stelios Katranidis & Theodore Panagiotidis & Costas Zontanos, 2012.
"An evaluation of the Greek Universities Economics Departments,"
Discussion Paper Series
2012_01, Department of Economics, University of Macedonia, revised Jan 2012.
- Stelios Katranidis & Theodore Panagiotidis & Costas Zontanos, 2012. "An Evaluation of the Greek Universities Economics Departments," Working Paper Series 03_12, The Rimini Centre for Economic Analysis.
- Frances P. Ruane & Richard S.J. Tol, 2007.
"Centres of Research Excellence in Economics in the Republic of Ireland,"
The Economic and Social Review,
Economic and Social Studies, vol. 38(3), pages 289â322.
- Frances Ruane & Richard S.J. Tol, 2007. "Centres of Research Excellence in Economics in the Republic of Ireland," Papers WP180, Economic and Social Research Institute (ESRI).
- Gianfranco Di Vaio & Daniel Waldenström & Jacob Weisdorf, 2009.
"Citation Success: Evidence from Economic History Journal Publications,"
10-01, University of Copenhagen. Department of Economics.
- Di Vaio, Gianfranco & Waldenström, Daniel & Weisdorf, Jacob, 2012. "Citation success: Evidence from economic history journal publications," Explorations in Economic History, Elsevier, vol. 49(1), pages 92-104.
- Waldenström, Daniel & Di Vaio, Gianfranco & Weisdorf, Jacob, 2010. "Citation Success: Evidence from Economic History Journal Publications," Working Paper Series 819, Research Institute of Industrial Economics, revised 20 Oct 2010.
- Gianfranco Di Vaio & Daniel Waldenström & Jacob Weisdorf, 2011. "Citation Success: Evidence from Economic History Journal Publications," Working Papers 0017, Utrecht University, Centre for Global Economic History.
- Müller, Harry, 2012. "Die Zitationshäufigkeit als Qualitätsindikator im Rahmen der Forschungsleistungsmessung," Discussion Papers of the Institute for Organisational Economics 1/2012, University of Münster, Institute for Organisational Economics.
- David L. Anderson & John Tressler, 2010. "The Merits of Using Citation-Based Journal Weighting Schemes to Measure Research Performance in Economics: The Case of New Zealand," Working Papers in Economics 10/03, University of Waikato, Department of Economics.
- David L. Anderson & John Tressler, 2011. "The Merits of Using Citations to Measure Research Output in Economics Departments: The New Zealand Case," Working Papers in Economics 11/11, University of Waikato, Department of Economics.
- Christian Seiler & Klaus Wohlrabe, 2011.
"Ranking Economists on the Basis of Many Indicators: An Alternative Approach Using RePEc Data,"
CESifo Working Paper Series
3691, CESifo Group Munich.
- Seiler, Christian & Wohlrabe, Klaus, 2012. "Ranking economists on the basis of many indicators: An alternative approach using RePEc data," Journal of Informetrics, Elsevier, vol. 6(3), pages 389-402.
- Matthias Meyer & Rüdiger W. Waldkirch & Michael A. Zaggl, 2012. "Relative Performance Measurement of Researchers: The Impact of Data Source Selection," Schmalenbach Business Review (sbr), LMU Munich School of Management, vol. 64(4), pages 308-330, October.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Helena Lundin).
If references are entirely missing, you can add them using this form.