IDEAS home Printed from https://ideas.repec.org/p/hhs/hastef/0693.html
   My bibliography  Save this paper

How Should Research Performance Be Measured? A Study of Swedish Economists

Author

Listed:

Abstract

Billions of euros are allocated every year to university research. Increased specialisation and international integration of research and researchers has sharply raised the need for comparisons of performance across fields, institutions and individual researchers. However, there is still no consensus regarding how such rankings should be conducted and what output measures are appropriate to use. We rank all full professors in a particular discipline, economics, in one European nation using seven established, and some of them commonly used, measures of research performance. Our examination shows both that the rank order can vary greatly across measures, and that depending on the measure used the distribution of total research out-put is valued very differently. The renowned KMS measure in economics stands out among the measures analysed here. It exhibits the weakest correlation with the others used in our study. We conclude by giving advice to funding councils and others assessing research quality on how to think about the use of both quantitative and qualitative measures of performance.

Suggested Citation

  • Henrekson, Magnus & Waldenström, Daniel, 2008. "How Should Research Performance Be Measured? A Study of Swedish Economists," SSE/EFI Working Paper Series in Economics and Finance 693, Stockholm School of Economics.
    • Handle: RePEc:hhs:hastef:0693
    as

    Download full text from publisher

    File URL: http://swopec.hhs.se/hastef/papers/hastef0693.pdf
    File Function: Complete Rendering
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. David N. Laband & Robert D. Tollison, 2003. "Dry Holes in Economic Research," Kyklos, Wiley Blackwell, vol. 56(2), pages 161-173, May.
    2. Kalaitzidakis, P. & Mamuneas, T.P. & Stengos, T., 2003. "Rankings of Academic Journals and Institutions," Working Papers 2003-8, University of Guelph, Department of Economics and Finance.
    3. Pantelis Kalaitzidakis & Theofanis P. Mamuneas & Thanasis Stengos, 2003. "Rankings of Academic Journals and Institutions in Economics," Journal of the European Economic Association, MIT Press, vol. 1(6), pages 1346-1366, December.
    4. Michel Lubrano & Luc Bauwens & Alan Kirman & Camelia Protopopescu, 2003. "Ranking Economics Departments in Europe: A Statistical Approach," Journal of the European Economic Association, MIT Press, vol. 1(6), pages 1367-1401, December.
    5. Richard Dusansky & Clayton J. Vernon, 1998. "Rankings of U.S. Economics Departments," Journal of Economic Perspectives, American Economic Association, vol. 12(1), pages 157-170, Winter.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ho Fai Chan & Bruno S. Frey & Jana Gallus & Markus Schaffner & Benno Torgler & Stephen Whyte, 2014. "Do the best scholars attract the highest speaking fees? An exploration of internal and external influence," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 793-817, October.
    2. Weilong Bi & Ho Fai Chan & Benno Torgler, 2019. "Self-esteem, self-symbolizing, and academic recognition: behavioral evidence from curricula vitae," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(1), pages 495-525, April.
    3. Müller, Harry, 2012. "Die Zitationshäufigkeit als Qualitätsindikator im Rahmen der Forschungsleistungsmessung," Discussion Papers of the Institute for Organisational Economics 1/2012, University of Münster, Institute for Organisational Economics.
    4. Rosalind S Hunter, 2009. "The Elite Brain Drain," Working Papers id:2048, eSocialSciences.
    5. Simon Ek & Magnus Henrekson, 2019. "The Geography and Concentration of Authorship in the Top Five: Implications For European Economics," Scottish Journal of Political Economy, Scottish Economic Society, vol. 66(2), pages 215-245, May.
    6. David L. Anderson & John Tressler, 2013. "The Relevance of the “h-” and “g-” Index to Economics in the Context of A Nation-Wide Research Evaluation Scheme: The New Zealand Case," Economic Papers, The Economic Society of Australia, vol. 32(1), pages 81-94, March.
    7. Di Vaio, Gianfranco & Waldenström, Daniel & Weisdorf, Jacob, 2012. "Citation success: Evidence from economic history journal publications," Explorations in Economic History, Elsevier, vol. 49(1), pages 92-104.
    8. Frances P. Ruane & Richard S.J. Tol, 2007. "Centres of Research Excellence in Economics in the Republic of Ireland," The Economic and Social Review, Economic and Social Studies, vol. 38(3), pages 289-322.
    9. John Tressler & David L. Anderson, 2012. "Citations as a Measure of the Research Outputs of New Zealand's Economics Departments: The Problem of 'Long and Variable Lags'," Agenda - A Journal of Policy Analysis and Reform, Australian National University, College of Business and Economics, School of Economics, vol. 19(1), pages 17-40.
    10. Matthias Meyer & Rüdiger W. Waldkirch & Michael A. Zaggl, 2012. "Relative Performance Measurement of Researchers: The Impact of Data Source Selection," Schmalenbach Business Review (sbr), LMU Munich School of Management, vol. 64(4), pages 308-330, October.
    11. Stelios Katranidis & Theodore Panagiotidis & Costas Zontanos, 2014. "An Evaluation Of The Greek Universities’ Economics Departments," Bulletin of Economic Research, Wiley Blackwell, vol. 66(2), pages 173-182, April.
    12. John Gibson, 2021. "The micro‐geography of academic research: How distinctive is economics?," Scottish Journal of Political Economy, Scottish Economic Society, vol. 68(4), pages 467-484, September.
    13. David L. Anderson & John Tressler, 2011. "The Merits of Using Citations to Measure Research Output in Economics Departments: The New Zealand Case," Working Papers in Economics 11/11, University of Waikato.
    14. Jelnov, Pavel & Weiss, Yoram, 2022. "Influence in economics and aging," Labour Economics, Elsevier, vol. 77(C).
    15. Seiler, Christian & Wohlrabe, Klaus, 2012. "Ranking economists on the basis of many indicators: An alternative approach using RePEc data," Journal of Informetrics, Elsevier, vol. 6(3), pages 389-402.
    16. Klaus Beckmann & Andrea Schneider, 2013. "The interaction of publications and appointments: new evidence on academic economists in Germany," Education Economics, Taylor & Francis Journals, vol. 21(4), pages 415-430, September.
    17. Raquel Carrasco & Javier Ruiz-Castillo, 2014. "The Evolution Of The Scientific Productivity Of Highly Productive Economists," Economic Inquiry, Western Economic Association International, vol. 52(1), pages 1-16, January.
    18. Seiler, Christian & Wohlrabe, Klaus, 2012. "Ranking economists on the basis of many indicators: An alternative approach using RePEc data," Journal of Informetrics, Elsevier, vol. 6(3), pages 389-402.
    19. Seyed Reza Mirnezami & Catherine Beaudry, 2016. "The effect of holding a research chair on scientists’ productivity," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(2), pages 399-454, May.
    20. David L. Anderson & John Tressler, 2010. "The Merits of Using Citation-Based Journal Weighting Schemes to Measure Research Performance in Economics: The Case of New Zealand," Working Papers in Economics 10/03, University of Waikato.
    21. Johansson, Dan & Karlsson, Johan & Malm, Arvid, 2020. "Family business—A missing link in economics?," Journal of Family Business Strategy, Elsevier, vol. 11(1).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Stephen Bazen & Patrick Moyes, 2012. "Elitism and stochastic dominance," Social Choice and Welfare, Springer;The Society for Social Choice and Welfare, vol. 39(1), pages 207-251, June.
    2. Jang C. Jin, 2009. "Publications in mathematical economics and econometrics: ranking of Asian universities and an application of Zipf's law," Asian-Pacific Economic Literature, The Crawford School, The Australian National University, vol. 23(2), pages 116-122, November.
    3. James B. Davies & Martin G. Kocher & Matthias Sutter, 2008. "Economics research in Canada: a long-run assessment of journal publications," Canadian Journal of Economics, Canadian Economics Association, vol. 41(1), pages 22-45, February.
    4. Baltagi, Badi H., 2007. "Worldwide Econometrics Rankings: 1989–2005," Econometric Theory, Cambridge University Press, vol. 23(5), pages 952-1012, October.
    5. Hendrik P. Dalen, 2021. "How the publish-or-perish principle divides a science: the case of economists," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1675-1694, February.
    6. Jin, Jang C. & Hong, Jin-Heon, 2008. "East Asian rankings of economics departments," Journal of Asian Economics, Elsevier, vol. 19(1), pages 74-82, February.
    7. van Dalen, Hendrik Peter, 2020. "How the Publish-or-Perish Principle Divides a Science : The Case of Academic Economists," Other publications TiSEM 6fbb6b92-0e06-4271-b6e7-3, Tilburg University, School of Economics and Management.
    8. Tombazos, Christis G., 2005. "A revisionist perspective of European research in economics," European Economic Review, Elsevier, vol. 49(2), pages 251-277, February.
    9. van Dalen, Hendrik Peter, 2021. "How the publish-or-perish principle divides a science: The case of economists," Other publications TiSEM a6a5a855-bb5a-4d52-a841-3, Tilburg University, School of Economics and Management.
    10. Tom Coupé, 2003. "Revealed Performances: Worldwide Rankings of Economists and Economics Departments, 1990-2000," Journal of the European Economic Association, MIT Press, vol. 1(6), pages 1309-1345, December.
    11. David L. Anderson & John Tressler, 2008. "Research Output in New Zealand Economics Department 2000-2006," Working Papers in Economics 08/05, University of Waikato.
    12. J. A. García & Rosa Rodriguez-Sánchez & J. Fdez-Valdivia, 2012. "A comparison of top economics departments in the US and EU on the basis of the multidimensional prestige of influential articles in 2010," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(3), pages 681-698, December.
    13. Raúl Ramos & Vicente Royuela & Jordi Suriñach, 2007. "An analysis of the determinants in Economics and Business publications by Spanish universities between 1994 and 2004," Scientometrics, Springer;Akadémiai Kiadó, vol. 71(1), pages 117-144, April.
    14. Frank Neri & Joan R. Rodgers, 2006. "Ranking Australian Economics Departments by Research Productivity," The Economic Record, The Economic Society of Australia, vol. 82(s1), pages 74-84, September.
    15. repec:cte:werepe:we063609 is not listed on IDEAS
    16. Ofer H. Azar & David M. Brock, 2008. "A Citation‐Based Ranking of Strategic Management Journals," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 17(3), pages 781-802, September.
    17. Rodgers, Joan R. & Valadkhani, Abbas, 2005. "Ranking of Australian Economics Departments Based on Their Total and Per Academic Staff Research Output," Economics Working Papers wp05-18, School of Economics, University of Wollongong, NSW, Australia.
    18. Derek Yu & Atoko Kasongo & Mariana Moses, 2017. "Examining the Performance of the South African Economics Departments, 2005-2014," South African Journal of Economics, Economic Society of South Africa, vol. 85(1), pages 138-158, March.
    19. David Anderson & John Tressler, 2008. "Research output in New Zealand economics departments 2000-2006: A stock approach," New Zealand Economic Papers, Taylor & Francis Journals, vol. 42(2), pages 155-189.
    20. Richard V. Adkisson, 2010. "Reptilian Economists of the World Unite: A Tolerance Manifesto," The American Economist, Sage Publications, vol. 55(2), pages 14-23, November.
    21. Javier Ruiz-Castillo, 2008. "Economics research in Spain during the 1990s: a literature review," Spanish Economic Review, Springer;Spanish Economic Association, vol. 10(3), pages 221-249, September.

    More about this item

    Keywords

    Impact of research; Ranking; Research productivity; Bibliometrics; Impact Factor;
    All these keywords.

    JEL classification:

    • A11 - General Economics and Teaching - - General Economics - - - Role of Economics; Role of Economists
    • A14 - General Economics and Teaching - - General Economics - - - Sociology of Economics
    • B41 - Schools of Economic Thought and Methodology - - Economic Methodology - - - Economic Methodology

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hhs:hastef:0693. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Helena Lundin (email available below). General contact details of provider: https://edirc.repec.org/data/erhhsse.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.