IDEAS home Printed from https://ideas.repec.org/p/ces/ifowps/_96.html
   My bibliography  Save this paper

Ranking Economists and Economic Institutions Using RePEc: Some Remarks

Author

Listed:
  • Christian Seiler
  • Klaus Wohlrabe

Abstract

In socio-economic sciences the RePEc network (Research Papers in Economics) has become an essential source both for the spread of existing and new economic research. Furthermore the calculation of rankings for authors and academic institutions play a central role. We provide some cautionary remarks on the ranking methodology employed by RePEc and show how the aggregated rankings maybe biased. Furthermore we offer a new ranking approach, based on standardization of scores, which allows interpersonal comparisons and is less sensitive to outliers. We illustrate our new approach with a large data set provided by RePEc based on 24,500 authors.

Suggested Citation

  • Christian Seiler & Klaus Wohlrabe, 2011. "Ranking Economists and Economic Institutions Using RePEc: Some Remarks," ifo Working Paper Series 96, ifo Institute - Leibniz Institute for Economic Research at the University of Munich.
  • Handle: RePEc:ces:ifowps:_96
    as

    Download full text from publisher

    File URL: https://www.ifo.de/DocDL/IfoWorkingPaper-96.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Christian Zimmermann, 2013. "Academic Rankings with RePEc," Econometrics, MDPI, Open Access Journal, vol. 1(3), pages 1-32, December.
    2. Dan Ben-David, 2010. "Ranking Israel’s economists," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(2), pages 351-364, February.
    3. Magnus Henrekson & Daniel Waldenström, 2011. "How Should Research Performance Be Measured? A Study Of Swedish Economists," Manchester School, University of Manchester, vol. 79(6), pages 1139-1156, December.
    4. Kalaitzidakis, P. & Mamuneas, T.P. & Stengos, T., 2003. "Rankings of Academic Journals and Institutions," Working Papers 2003-8, University of Guelph, Department of Economics and Finance.
    5. Norris, Michael & Oppenheim, Charles, 2007. "Comparing alternatives to the Web of Science for coverage of the social sciences’ literature," Journal of Informetrics, Elsevier, vol. 1(2), pages 161-169.
    6. Gonzalez-Navarro, Marco & Quintana-Domeque, Climent, 2010. "Urban Infrastructure and Economic Development: Experimental Evidence from Street Pavement," IZA Discussion Papers 5346, Institute of Labor Economics (IZA).
    7. Ortega, José Luis & López-Romero, Elena & Fernández, Inés, 2011. "Multivariate approach to classify research institutes according to their outputs: The case of the CSIC's institutes," Journal of Informetrics, Elsevier, vol. 5(3), pages 323-332.
    8. Rolf Ketzler & Klaus F. Zimmermann, 2009. "Publications: German economic research institutes on track," Scientometrics, Springer;Akadémiai Kiadó, vol. 80(1), pages 231-252, July.
    9. Graves, Philip E & Marchand, James R & Thompson, Randal, 1982. "Economics Departmental Rankings: Research Incentives, Constraints, and Efficiency," American Economic Review, American Economic Association, vol. 72(5), pages 1131-1141, December.
    10. Pantelis Kalaitzidakis & Theofanis P. Mamuneas & Thanasis Stengos, 2003. "Rankings of Academic Journals and Institutions in Economics," Journal of the European Economic Association, MIT Press, vol. 1(6), pages 1346-1366, December.
    11. Liebowitz, S J & Palmer, J P, 1984. "Assessing the Relative Impacts of Economic Journals," Journal of Economic Literature, American Economic Association, vol. 22(1), pages 77-88, March.
    12. Angelina Keil & Peter Huber, 2004. "„Wo die Luft dünn wird…”– Zur Publikationstätigkeit der Wirtschaftsforschungsinstitute Österreichs und Deutschlands," Perspektiven der Wirtschaftspolitik, Verein für Socialpolitik, vol. 5(3), pages 363-375, August.
    13. Bruno S. Frey & Reiner Eichenberger, 1993. "American and European Economics and Economists," Journal of Economic Perspectives, American Economic Association, vol. 7(4), pages 185-193, Fall.
    14. Baltagi, Badi H, 1999. "Applied Econometrics Rankings: 1989-1995," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 14(4), pages 423-441, July-Aug..
    15. repec:ebl:ecbull:v:1:y:2004:i:4:p:1-19 is not listed on IDEAS
    16. Juan Dolado & Antonio García-Romero & Gema Zamarro, 2003. "Publishing performance in economics: Spanish rankings (1990-1999)," Spanish Economic Review, Springer;Spanish Economic Association, vol. 5(4), pages 317-317, November.
    17. Martin Steininger & Bernd Süssmuth, 2005. "Elfenbeinligen und ihre Erfassung: Ein Kommentar und eine neuerliche Messung der Publikationstätigkeit der Wirtschaftsforschungsinstitute im deutschsprachigen Raum: 1989–2003," Perspektiven der Wirtschaftspolitik, Verein für Socialpolitik, vol. 6(3), pages 409-420, August.
    18. Courtault, Jean-Michel & Hayek, Naïla & Rimbaux, Eric & Zhu, Tong, 2010. "Research in economics and management in France: A bibliometric study using the h-index," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 39(2), pages 329-337, April.
    19. Frances P. Ruane & Richard S. J. Tol, 2009. "A Hirsch measure for the quality of research supervision, and an illustration with trade economists," Scientometrics, Springer;Akadémiai Kiadó, vol. 80(3), pages 613-624, September.
    20. Richard Dusansky & Clayton J. Vernon, 1998. "Rankings of U.S. Economics Departments," Journal of Economic Perspectives, American Economic Association, vol. 12(1), pages 157-170, Winter.
    21. Kalaitzidakis, Pantelis & Mamuneas, Theofanis P. & Stengos, Thanasis, 1999. "European economics: An analysis based on publications in the core journals," European Economic Review, Elsevier, vol. 43(4-6), pages 1150-1168, April.
    22. Robert Hofmeister & Heinrich W. Ursprung, 2008. "Das Handelsblatt Ökonomen‐Ranking 2007: Eine kritische Beurteilung," Perspektiven der Wirtschaftspolitik, Verein für Socialpolitik, vol. 9(3), pages 254-266, August.
    23. Glenn Ellison, 2002. "The Slowdown of the Economics Publishing Process," Journal of Political Economy, University of Chicago Press, vol. 110(5), pages 947-993, October.
    24. Kirman, Alan & Dahl, Mogens, 1994. "Economic research in Europe," European Economic Review, Elsevier, vol. 38(3-4), pages 505-522, April.
    25. Baltagi, Badi H., 2003. "Worldwide Institutional And Individual Rankings In Econometrics Over The Period 1989–1999: An Update," Econometric Theory, Cambridge University Press, vol. 19(1), pages 165-224, February.
    26. Costas, Rodrigo & Bordons, María, 2007. "The h-index: Advantages, limitations and its relation with other bibliometric indicators at the micro level," Journal of Informetrics, Elsevier, vol. 1(3), pages 193-203.
    27. repec:jns:jbstat:v:227:y:2007:i:2:p:187-208 is not listed on IDEAS
    28. Pierre-Philippe Combes & Laurent Linnemer, 2003. "Where are the Economists Who Publish? Publication Concentration and Rankings in Europe Based on Cumulative Publications," Journal of the European Economic Association, MIT Press, vol. 1(6), pages 1250-1308, December.
    29. David L. Anderson & John Tressler, 2008. "Research Output in New Zealand Economics Department 2000-2006," Working Papers in Economics 08/05, University of Waikato.
    30. Joseph Macri & Dipendra Sinha, 2004. "Rankings of Economists in Teaching Economics Departments in Australia, 1988-2000," Economics Bulletin, AccessEcon, vol. 1(4), pages 1-19.
    31. Alonso, S. & Cabrerizo, F.J. & Herrera-Viedma, E. & Herrera, F., 2009. "h-Index: A review focused in its variants, computation and standardization for different scientific fields," Journal of Informetrics, Elsevier, vol. 3(4), pages 273-289.
    32. Frances Ruane & Richard S.J. Tol, 2007. "Refined (Successive) H-Indices: An Application To Economics In The Republic Of Ireland," Working Papers FNU-130, Research unit Sustainability and Global Change, Hamburg University, revised Mar 2007.
    33. Christian Seiler & Klaus Wohlrabe, 2010. "RePEc – An Independent Platform for Measuring Output in Economics," CESifo Forum, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, vol. 11(04), pages 72-77, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Christian Seiler & Klaus Wohlrabe, 2011. "Ranking Economists on the Basis of Many Indicators: An Alternative Approach Using RePEc Data," CESifo Working Paper Series 3691, CESifo.
    2. Joseph Macri & Dipendra Sinha, 2006. "Rankings Methodology for International Comparisons of Institutions and Individuals: an Application to Economics in Australia and New Zealand," Journal of Economic Surveys, Wiley Blackwell, vol. 20(1), pages 111-156, February.
    3. Konstantinos Chatzimichael & Pantelis Kalaitzidakis & Vangelis Tzouvelekas, 2017. "Measuring the publishing productivity of economics departments in Europe," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 889-908, November.
    4. Baltagi, Badi H., 2007. "Worldwide Econometrics Rankings: 1989–2005," Econometric Theory, Cambridge University Press, vol. 23(5), pages 952-1012, October.
    5. David L. Anderson & John Tressler, 2017. "Researcher rank stability across alternative output measurement schemes in the context of a time limited research evaluation: the New Zealand case," Applied Economics, Taylor & Francis Journals, vol. 49(45), pages 4542-4553, September.
    6. Lutz Bornmann & Alexander Butz & Klaus Wohlrabe, 2018. "What are the top five journals in economics? A new meta-ranking," Applied Economics, Taylor & Francis Journals, vol. 50(6), pages 659-675, February.
    7. David L. Anderson & John Tressler, 2009. "The Excellence in Research for Australia Scheme: An Evaluation of the Draft Journal Weights for Economics," Working Papers in Economics 09/07, University of Waikato.
    8. Jang C. Jin, 2009. "Publications in mathematical economics and econometrics: ranking of Asian universities and an application of Zipf's law," Asian-Pacific Economic Literature, Asia Pacific School of Economics and Government, The Australian National University, vol. 23(2), pages 116-122, November.
    9. James B. Davies & Martin G. Kocher & Matthias Sutter, 2008. "Economics research in Canada: a long-run assessment of journal publications," Canadian Journal of Economics, Canadian Economics Association, vol. 41(1), pages 22-45, February.
    10. Tilak Mukhopadhyay & Subrata Sarar, 2010. "Rankings of Economics Journals and Departments in India," Indira Gandhi Institute of Development Research, Mumbai Working Papers 2010-021, Indira Gandhi Institute of Development Research, Mumbai, India.
    11. Ana Rute Cardoso & Paulo Guimarães & Klaus F. Zimmermann, 2010. "Comparing the early research performance of PhD graduates in labor economics in Europe and the USA," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(3), pages 621-637, September.
    12. Stelios Katranidis & Theodore Panagiotidis & Costas Zontanos, 2014. "An Evaluation Of The Greek Universities’ Economics Departments," Bulletin of Economic Research, Wiley Blackwell, vol. 66(2), pages 173-182, April.
    13. Javier Ruiz-Castillo, 2008. "Economics research in Spain during the 1990s: a literature review," Spanish Economic Review, Springer;Spanish Economic Association, vol. 10(3), pages 221-249, September.
    14. Joseph Macri & Michael McAleer & Dipendra Sinha, 2009. "On the Robustness of Alternative Rankings Methodologies For Australian and New Zealand Economics Departments," CIRJE F-Series CIRJE-F-660, CIRJE, Faculty of Economics, University of Tokyo.
    15. David Anderson & John Tressler, 2008. "Research output in New Zealand economics departments 2000-2006: A stock approach," New Zealand Economic Papers, Taylor & Francis Journals, vol. 42(2), pages 155-189.
    16. David L. Anderson & John Tressler, 2013. "The Relevance of the “h-” and “g-” Index to Economics in the Context of A Nation-Wide Research Evaluation Scheme: The New Zealand Case," Economic Papers, The Economic Society of Australia, vol. 32(1), pages 81-94, March.
    17. Jan Ours & Frederic Vermeulen, 2007. "Ranking Dutch Economists," De Economist, Springer, vol. 155(4), pages 469-487, December.
    18. Issler, João Victor & Ferreira, Rachel Couto, 2003. "Avaliando pesquisadores e departamentos de economia no Brasil a partir de citações internacionais," FGV EPGE Economics Working Papers (Ensaios Economicos da EPGE) 500, EPGE Brazilian School of Economics and Finance - FGV EPGE (Brazil).
    19. Murat Çokgezen, 2019. "Research Performance of Turkish Economists and Economics Departments: Another Update and a Review of the 2000s," Emerging Markets Finance and Trade, Taylor & Francis Journals, vol. 55(1), pages 133-149, January.
    20. Ana Rute Cardoso & Paulo Guimarães & Klaus F. Zimmermann, 2010. "Trends in Economic Research: An International Perspective," Kyklos, Wiley Blackwell, vol. 63(4), pages 479-494, November.

    More about this item

    Keywords

    Rankings; RePEc; ranking aggregation; standardization;
    All these keywords.

    JEL classification:

    • A12 - General Economics and Teaching - - General Economics - - - Relation of Economics to Other Disciplines
    • A14 - General Economics and Teaching - - General Economics - - - Sociology of Economics

    Lists

    This item is featured on the following reading lists, Wikipedia, or ReplicationWiki pages:
    1. Papers using RePEc data

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ces:ifowps:_96. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/ifooode.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Klaus Wohlrabe (email available below). General contact details of provider: https://edirc.repec.org/data/ifooode.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.