IDEAS home Printed from https://ideas.repec.org/p/knz/dpteco/1002.html
   My bibliography  Save this paper

Research evaluation and journal quality weights: Much ado about nothing?

Author

Listed:
  • Matthias Krapf

    () (Department of Economics, University of Konstanz, Germany)

Abstract

Research evaluations based on quality weighted publication output are often criticized on account of the employed journal quality weights. This study shows that evaluations of entire research organizations are very robust with respect to the choice of readily available weighting schemes. We document this robustness by applying rather different weighting schemes to otherwise identical rankings. Our unit of analysis consists of German, Austrian and Swiss university departments in business administration and economics.

Suggested Citation

  • Matthias Krapf, 2010. "Research evaluation and journal quality weights: Much ado about nothing?," Working Paper Series of the Department of Economics, University of Konstanz 2010-02, Department of Economics, University of Konstanz.
  • Handle: RePEc:knz:dpteco:1002
    as

    Download full text from publisher

    File URL: http://www.uni-konstanz.de/FuF/wiwi/workingpaperseries/WP_Krapf-2-10.pdf
    Download Restriction: no

    References listed on IDEAS

    as
    1. Oliver Fabel & Miriam Hein & Robert Hofmeister, 2008. "Research Productivity in Business Economics: An Investigation of Austrian, German and Swiss Universities," German Economic Review, Verein für Socialpolitik, vol. 9, pages 506-531, November.
    2. Michael Rauber & Heinrich W. Ursprung, 2008. "Life Cycle and Cohort Productivity in Economic Research: The Case of Germany," German Economic Review, Verein für Socialpolitik, vol. 9, pages 431-456, November.
    3. Ignacio Palacios-Huerta & Oscar Volij, 2004. "The Measurement of Intellectual Influence," Econometrica, Econometric Society, vol. 72(3), pages 963-977, May.
    4. Klaus Ritzberger, 2008. "A Ranking of Journals in Economics and Related Fields," German Economic Review, Verein für Socialpolitik, vol. 9, pages 402-430, November.
    5. Günther G. Schulze & Susanne Warning & Christian Wiermann, 2008. "Zeitschriftenrankings für die Wirtschaftswissenschaften - Konstruktion eines umfassenden Metaindexes," Perspektiven der Wirtschaftspolitik, Verein für Socialpolitik, vol. 9(3), pages 286-305, August.
    6. Kalaitzidakis, P. & Mamuneas, T.P. & Stengos, T., 2003. "Rankings of Academic Journals and Institutions," Working Papers 2003-8, University of Guelph, Department of Economics and Finance.
    7. Friedrich Schneider & Heinrich W. Ursprung, 2008. "The 2008 GEA Journal-Ranking for the Economics Profession," German Economic Review, Verein für Socialpolitik, vol. 9, pages 532-538, November.
    8. Pantelis Kalaitzidakis & Theofanis P. Mamuneas & Thanasis Stengos, 2003. "Rankings of Academic Journals and Institutions in Economics," Journal of the European Economic Association, MIT Press, vol. 1(6), pages 1346-1366, December.
    9. Robert Hofmeister & Heinrich W. Ursprung, 2008. "Das Handelsblatt Ökonomen-Ranking 2007: Eine kritische Beurteilung," Perspektiven der Wirtschaftspolitik, Verein für Socialpolitik, vol. 9(3), pages 254-266, August.
    10. Partha, Dasgupta & David, Paul A., 1994. "Toward a new economics of science," Research Policy, Elsevier, vol. 23(5), pages 487-521, September.
    11. repec:jns:jbstat:v:227:y:2007:i:2:p:187-208 is not listed on IDEAS
    12. Pierre-Philippe Combes & Laurent Linnemer, 2003. "Where are the Economists Who Publish? Publication Concentration and Rankings in Europe Based on Cumulative Publications," Journal of the European Economic Association, MIT Press, vol. 1(6), pages 1250-1308, December.
    13. Pierre-Philippe Combes & Laurent Linnemer, 2010. "Inferring Missing Citations: A Quantitative Multi-Criteria Ranking of all Journals in Economics," Working Papers halshs-00520325, HAL.
    14. Michael Rauber & Heinrich W. Ursprung, 2008. "Evaluation of Researchers: A Life Cycle Analysis of German Academic Economists," Conferences on New Political Economy,in: Max Albert & Stefan Voigt & Dieter Schmidtchen (ed.), Conferences on New Political Economy, edition 1, volume 25, pages 101-122(2 Mohr Siebeck, Tübingen.
    15. Friedrich Schneider, 2008. "Special Issue on the Economics of the Economics Profession: Editorial," German Economic Review, Verein für Socialpolitik, vol. 9, pages 397-401, November.
    16. Ursprung Heinrich W. & Zimmer Markus, 2007. "Who is the ”Platz-Hirsch“ of the German Economics Profession?: A Citation Analysis," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 227(2), pages 187-208, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. J. Paul Elhorst & Katarina Zigova, 2011. "Evidence of Competition in Research Activity among Economic Department using Spatial Econometric Techniques," Working Paper Series of the Department of Economics, University of Konstanz 2011-04, Department of Economics, University of Konstanz.
    2. Tobias Kollmann & Andreas Kuckertz & Christoph Stöckmann & Patrick Krell, 2012. "Die Entrepreneurshipforschung in Deutschland, Österreich und der Schweiz. Eine Resonanzanalyse," ZfKE – Zeitschrift für KMU und Entrepreneurship, Duncker & Humblot, Berlin, vol. 60(1), pages 53-76.
    3. Ioana Alexandra Horodnic & Adriana Zaiţ, 2015. "Motivation and research productivity in a university system undergoing transition," Research Evaluation, Oxford University Press, vol. 24(3), pages 282-292.
    4. Ioana Alexandra HORODNIC, 2014. "Academic Performance: Measurement Methods Used In Socio - Economic Sciences," THE YEARBOOK OF THE “GH. ZANE” INSTITUTE OF ECONOMIC RESEARCHES, Gheorghe Zane Institute for Economic and Social Research ( from THE ROMANIAN ACADEMY, JASSY BRANCH), vol. 23(1), pages 5-17.
    5. Agnes Bäker & Susanne Breuninger & Julia Muschallik & Kerstin Pull & Uschi Backes-Gellner, 2016. "Time to Go? (Inter)National Mobility and Appointment Success of Young Academics," Schmalenbach Business Review, Springer;Schmalenbach-Gesellschaft, vol. 17(3), pages 401-421, December.
    6. Matthias Meyer & Rüdiger W. Waldkirch & Michael A. Zaggl, 2012. "Relative Performance Measurement of Researchers: The Impact of Data Source Selection," Schmalenbach Business Review (sbr), LMU Munich School of Management, vol. 64(4), pages 308-330, October.
    7. repec:bla:germec:v:18:y:2017:i:4:p:492-515 is not listed on IDEAS
    8. repec:eme:aecozz:s0731-905320160000037016 is not listed on IDEAS
    9. Peter Burridge & J. Paul Elhorst & Katarina Zigova, 2016. "Group Interaction in Research and the Use of General Nesting Spatial Models," Advances in Econometrics,in: Spatial Econometrics: Qualitative and Limited Dependent Variables, volume 37, pages 223-258 Emerald Publishing Ltd.
    10. repec:sbe:breart:v:31:y:2011:i:1:a:4060 is not listed on IDEAS
    11. Julia Muschallik & Kerstin Pull, 2016. "Mentoring in higher education: does it enhance mentees’ research productivity?," Education Economics, Taylor & Francis Journals, vol. 24(2), pages 210-223, April.
    12. Adam Ayaita & Kerstin Pull & Uschi Backes-Gellner, 2017. "You get what you 'pay' for: Academic attention, career incentives and changes in publication portfolios of business and economics researchers," Economics of Education Working Paper Series 0133, University of Zurich, Department of Business Administration (IBW), revised Sep 2017.

    More about this item

    Keywords

    Research evaluation; university management;

    JEL classification:

    • I20 - Health, Education, and Welfare - - Education - - - General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:knz:dpteco:1002. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Office Ursprung) or (Rebekah McClure). General contact details of provider: http://edirc.repec.org/data/fwkonde.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.