IDEAS home Printed from https://ideas.repec.org/a/sae/evarev/v25y2001i5p555-566.html
   My bibliography  Save this article

Tools for Evaluating Research Output

Author

Listed:
  • Matthias Sutter

    (University of Innsbruck)

  • Martin G. Kocher

    (University of Innsbruck)

Abstract

Journal rankings are important for evaluating research output, for academic promotions, and for allocating funds. Examining the assignment of economics journals to different quartiles of citation-based rankings, the authors found that about 60% of journals remain in the same quartile and about 95% of journals remain in the same or move in a neighborhood quartile within 5- to 10-year intervals. Top journal rankings exhibit even more inertia, which is a desirable property for decisions on promotion, tenure, or institutional evaluation. A reexamination of journal assignment to different quality categories every 5 or 10 years, therefore, suffices for practical evaluation purposes.

Suggested Citation

  • Matthias Sutter & Martin G. Kocher, 2001. "Tools for Evaluating Research Output," Evaluation Review, , vol. 25(5), pages 555-566, October.
    • Handle: RePEc:sae:evarev:v:25:y:2001:i:5:p:555-566
    DOI: 10.1177/0193841X0102500503
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0193841X0102500503
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0193841X0102500503?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Michael Bräuninger & Justus Haucap, 2001. "Was Ökonomen lesen und schätzen: Ergebnisse einer Umfrage," Perspektiven der Wirtschaftspolitik, Verein für Socialpolitik, vol. 2(2), pages 185-210, May.
    2. Kalaitzidakis, Pantelis & Mamuneas, Theofanis P. & Stengos, Thanasis, 1999. "European economics: An analysis based on publications in the core journals," European Economic Review, Elsevier, vol. 43(4-6), pages 1150-1168, April.
    3. Elliott, Caroline & Greenaway, David & Sapsford, David, 1998. "Who's publishing who? The national composition of contributors to some core US and European journals," European Economic Review, Elsevier, vol. 42(1), pages 201-206, January.
    4. Clive Beed & Cara Beed, 1996. "Measuring the Quality of Academic Journals: The Case of Economics," Journal of Post Keynesian Economics, Taylor & Francis Journals, vol. 18(3), pages 369-396, March.
    5. Hodgson, Geoffrey M & Rothman, Harry, 1999. "The Editors and Authors of Economics Journals: A Case of Institutional Oligopoly?," Economic Journal, Royal Economic Society, vol. 109(453), pages 165-186, February.
    6. Burton, M P & Phimister, Euan, 1995. "Core Journals: A Reappraisal of the Diamond List," Economic Journal, Royal Economic Society, vol. 105(429), pages 361-373, March.
    7. Kocher, Martin G & Sutter, Matthias, 2001. "The Institutional Concentration of Authors in Top Journals of Economics during the Last Two Decades," Economic Journal, Royal Economic Society, vol. 111(472), pages 405-421, June.
    8. Laband, David N & Piette, Michael J, 1994. "The Relative Impacts of Economics Journals: 1970-1990," Journal of Economic Literature, American Economic Association, vol. 32(2), pages 640-666, June.
    9. Liebowitz, S J & Palmer, J P, 1984. "Assessing the Relative Impacts of Economic Journals," Journal of Economic Literature, American Economic Association, vol. 22(1), pages 77-88, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kocher, Martin G. & Luptacik, Mikulas & Sutter, Matthias, 2006. "Measuring productivity of research in economics: A cross-country study using DEA," Socio-Economic Planning Sciences, Elsevier, vol. 40(4), pages 314-332, December.
    2. Michael Rauber & Heinrich W. Ursprung, 2008. "Life Cycle and Cohort Productivity in Economic Research: The Case of Germany," German Economic Review, Verein für Socialpolitik, vol. 9(4), pages 431-456, November.
    3. Michael Bräuninger & Justus Haucap, 2001. "Was Ökonomen lesen und schätzen: Ergebnisse einer Umfrage," Perspektiven der Wirtschaftspolitik, Verein für Socialpolitik, vol. 2(2), pages 185-210, May.
    4. Ho Fai Chan & Bruno S. Frey & Jana Gallus & Benno Torgler, 2013. "Does the John Bates Clark Medal boost subsequent productivity and citation success?," ECON - Working Papers 111, Department of Economics - University of Zurich.
    5. Michael Rauber & Heinrich W. Ursprung, 2008. "Life Cycle and Cohort Productivity in Economic Research: The Case of Germany," German Economic Review, Verein für Socialpolitik, vol. 9(4), pages 431-456, November.
    6. James B. Davies & Martin G. Kocher & Matthias Sutter, 2008. "Economics research in Canada: a long-run assessment of journal publications," Canadian Journal of Economics, Canadian Economics Association, vol. 41(1), pages 22-45, February.
    7. Yi-Ching Liaw & Te-Yi Chan & Chin-Yuan Fan & Cheng-Hsin Chiang, 2014. "Can the technological impact of academic journals be evaluated? The practice of non-patent reference (NPR) analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 17-37, October.
    8. Justus Haucap & Johannes Muck, 2015. "What drives the relevance and reputation of economics journals? An update from a survey among economists," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(3), pages 849-877, June.
    9. Ho F. Chan & Franklin G. Mixon & Benno Torgler, 2018. "Relation of early career performance and recognition to the probability of winning the Nobel Prize in economics," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 1069-1086, March.
    10. Matthias Sutter & Martin Kocher & Robert Mrsic, 2002. "Representation and Educational Background of European Economists in Top Journals of Economics," Empirica, Springer;Austrian Institute for Economic Research;Austrian Economic Association, vol. 29(4), pages 275-288, December.
    11. David L. Anderson & John Tressler, 2017. "Researcher rank stability across alternative output measurement schemes in the context of a time limited research evaluation: the New Zealand case," Applied Economics, Taylor & Francis Journals, vol. 49(45), pages 4542-4553, September.
    12. Pajić, Dejan, 2015. "On the stability of citation-based journal rankings," Journal of Informetrics, Elsevier, vol. 9(4), pages 990-1006.
    13. Frandsen, Tove Faber & Nicolaisen, Jeppe, 2010. "What is in a name? Credit assignment practices in different disciplines," Journal of Informetrics, Elsevier, vol. 4(4), pages 608-617.
    14. Chan, Ho Fai & Frey, Bruno S. & Gallus, Jana & Torgler, Benno, 2014. "Academic honors and performance," Labour Economics, Elsevier, vol. 31(C), pages 188-204.
    15. Ho Fai Chan & Bruno S. Frey & Jana Gallus & Benno Torgler, 2013. "Does the John Bates Clark Medal boost subsequent productivity and citation success?," ECON - Working Papers 111, Department of Economics - University of Zurich.
    16. Justus Haucap & Tobias Hartwich & André Uhde, 2005. "Besonderheiten und Wettbewerbsprobleme des Marktes für wissenschaftliche Fachzeitschriften," Vierteljahrshefte zur Wirtschaftsforschung / Quarterly Journal of Economic Research, DIW Berlin, German Institute for Economic Research, vol. 74(3), pages 85-107.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kocher, Martin G & Sutter, Matthias, 2001. "The Institutional Concentration of Authors in Top Journals of Economics during the Last Two Decades," Economic Journal, Royal Economic Society, vol. 111(472), pages 405-421, June.
    2. Matthias Sutter & Martin G. Kochner, 2001. "Power laws of research output. Evidence for journals of economics," Scientometrics, Springer;Akadémiai Kiadó, vol. 51(2), pages 405-414, June.
    3. Cloos, Janis & Greiff, Matthias & Rusch, Hannes, 2020. "Geographical Concentration and Editorial Favoritism within the Field of Laboratory Experimental Economics (RM/19/029-revised-)," Research Memorandum 014, Maastricht University, Graduate School of Business and Economics (GSBE).
    4. Kocher, Martin G. & Luptacik, Mikulas & Sutter, Matthias, 2006. "Measuring productivity of research in economics: A cross-country study using DEA," Socio-Economic Planning Sciences, Elsevier, vol. 40(4), pages 314-332, December.
    5. Mirucki, Jean, 2009. "Assessing trends in editorial preferences towards 'Industrial Organization' articles: 1991-2000," MPRA Paper 31168, University Library of Munich, Germany, revised Dec 2009.
    6. Michael Bräuninger & Justus Haucap, 2003. "Reputation and Relevance of Economics Journals," Kyklos, Wiley Blackwell, vol. 56(2), pages 175-197, May.
    7. Mirucki, Jean, 2007. "Visibilité des contributions en économie industrielle dans les revues scientifiques internationales [Visibility of contributions in Industrial Organization in international scientific journals]," MPRA Paper 31688, University Library of Munich, Germany, revised Jun 2007.
    8. Mirucki, Jean, 2009. "Assessing trends in editorial preferences towards industrial organization article," MPRA Paper 30406, University Library of Munich, Germany, revised Dec 2009.
    9. Joseph Macri & Dipendra Sinha, 2006. "Rankings Methodology for International Comparisons of Institutions and Individuals: an Application to Economics in Australia and New Zealand," Journal of Economic Surveys, Wiley Blackwell, vol. 20(1), pages 111-156, February.
    10. Matthias Sutter & Martin Kocher & Robert Mrsic, 2002. "Representation and Educational Background of European Economists in Top Journals of Economics," Empirica, Springer;Austrian Institute for Economic Research;Austrian Economic Association, vol. 29(4), pages 275-288, December.
    11. Cloos, Janis & Greiff, Matthias & Rusch, Hannes, 2019. "Geographical Concentration and Editorial Favoritism within the Field of Laboratory Experimental Economics," Research Memorandum 029, Maastricht University, Graduate School of Business and Economics (GSBE).
    12. Wohlrabe, Klaus, 2016. "Taking the Temperature: A Meta-Ranking of Economics Journals," MPRA Paper 68933, University Library of Munich, Germany.
    13. Konstantinos Chatzimichael & Pantelis Kalaitzidakis & Vangelis Tzouvelekas, 2017. "Measuring the publishing productivity of economics departments in Europe," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 889-908, November.
    14. Thorsten Hennig-Thurau & Gianfranco Walsh & Ulf Schrader, 2004. "VHB-JOURQUAL: Ein Ranking von betriebswirtschaftlich-relevanten Zeitschriften auf der Grundlage von Expertenurteilen," Schmalenbach Journal of Business Research, Springer, vol. 56(6), pages 520-545, September.
    15. Paul Gopuran Devassy Bino & Sasidharan Subash & Ananthakrishnan Ramanathan, 2005. "Concentration in Knowledge Output: A case of Economics Journals," European Journal of Comparative Economics, Cattaneo University (LIUC), vol. 2(2), pages 261-279, December.
    16. Frandsen, Tove Faber & Nicolaisen, Jeppe, 2010. "What is in a name? Credit assignment practices in different disciplines," Journal of Informetrics, Elsevier, vol. 4(4), pages 608-617.
    17. Lutz Bornmann & Alexander Butz & Klaus Wohlrabe, 2018. "What are the top five journals in economics? A new meta-ranking," Applied Economics, Taylor & Francis Journals, vol. 50(6), pages 659-675, February.
    18. Jean Mirucki, 2003. "Assessing editorial preferences towards Industrial Organization articles," Applied Economics, Taylor & Francis Journals, vol. 35(2), pages 239-243.
    19. Martin Macháček, 2004. "Komparace tematické struktury časopiseckých publikací českých a evropských ekonomů (1999 - 2002) [Comparison of the thematic structure of journal articles by czech and european economists]," Politická ekonomie, Prague University of Economics and Business, vol. 2004(1), pages 74-90.
    20. Justus Haucap & Johannes Muck, 2015. "What drives the relevance and reputation of economics journals? An update from a survey among economists," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(3), pages 849-877, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:evarev:v:25:y:2001:i:5:p:555-566. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.