IDEAS home Printed from https://ideas.repec.org/a/ces/ifosdt/v63y2010i21p18-28.html
   My bibliography  Save this article

Quality analysis of journals in economics - quotation databases and impact factors in the online age

Author

Listed:
  • Karol Paludkiewicz
  • Klaus Wohlrabe

Abstract

The appraisal of the quality of scientific research is important both for the scientific community and for the public. Despite all criticism a quantitative assessment is necessary. In science, quotations and the impact factor derived from them play an important role. This article presents three quotation databases and how they operate: the Web of Science, which contains the Social Science Citation Index, Google Scholar and RePEc. In addition, the advantages and disadvantages of the impact factors, one of the most important measurements for scientific journals, are presented. With regard to economics, the RePEc network is certainly the most suitable forum for the assessment of (quantitative) measurements both for journals and for working-paper series. However, a complete inclusion of quotations has not yet been reached so that all figures must still be considered experimental.

Suggested Citation

  • Karol Paludkiewicz & Klaus Wohlrabe, 2010. "Quality analysis of journals in economics - quotation databases and impact factors in the online age," ifo Schnelldienst, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, vol. 63(21), pages 18-28, November.
    • Handle: RePEc:ces:ifosdt:v:63:y:2010:i:21:p:18-28
    as

    Download full text from publisher

    File URL: https://www.ifo.de/DocDL/ifosd_2010_21_3.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Glenn Ellison, 2013. "How Does the Market Use Citation Data? The Hirsch Index in Economics," American Economic Journal: Applied Economics, American Economic Association, vol. 5(3), pages 63-90, July.
    2. Christian Seiler & Klaus Wohlrabe, 2010. "RePEc - an independent platform to measure research output in economics," ifo Schnelldienst, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, vol. 63(07), pages 43-48, April.
    3. Bar-Ilan, Judit & Levene, Mark & Lin, Ayelet, 2007. "Some measures for comparing citation databases," Journal of Informetrics, Elsevier, vol. 1(1), pages 26-34.
    4. Per O. Seglen, 1994. "Causal relationship between article citedness and journal impact," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 45(1), pages 1-11, January.
    5. Richard S. J. Tol, 2009. "The h-index and its alternatives: An application to the 100 most prolific economists," Scientometrics, Springer;Akadémiai Kiadó, vol. 80(2), pages 317-324, August.
    6. Seiler, Christian & Wohlrabe, Klaus, 2012. "Ranking economists on the basis of many indicators: An alternative approach using RePEc data," Journal of Informetrics, Elsevier, vol. 6(3), pages 389-402.
    7. Glenn Ellison, 2002. "The Slowdown of the Economics Publishing Process," Journal of Political Economy, University of Chicago Press, vol. 110(5), pages 947-993, October.
    8. Klaus Ritzberger, 2008. "A Ranking of Journals in Economics and Related Fields," German Economic Review, Verein für Socialpolitik, vol. 9, pages 402-430, November.
    9. Günther G. Schulze & Susanne Warning & Christian Wiermann, 2008. "Zeitschriftenrankings für die Wirtschaftswissenschaften – Konstruktion eines umfassenden Metaindexes," Perspektiven der Wirtschaftspolitik, Verein für Socialpolitik, vol. 9(3), pages 286-305, August.
    10. Bruno S. Frey & Katja Rost, 2010. "Do rankings reflect research quality?," Journal of Applied Economics, Universidad del CEMA, vol. 13, pages 1-38, May.
    11. Norris, Michael & Oppenheim, Charles, 2007. "Comparing alternatives to the Web of Science for coverage of the social sciences’ literature," Journal of Informetrics, Elsevier, vol. 1(2), pages 161-169.
    12. repec:jns:jbstat:v:227:y:2007:i:2:p:187-208 is not listed on IDEAS
    13. Pierre-Philippe Combes & Laurent Linnemer, 2010. "Inferring Missing Citations: A Quantitative Multi-Criteria Ranking of all Journals in Economics," Working Papers halshs-00520325, HAL.
    14. Anton J. Nederhof, 2006. "Bibliometric monitoring of research performance in the Social Sciences and the Humanities: A Review," Scientometrics, Springer;Akadémiai Kiadó, vol. 66(1), pages 81-100, January.
    15. Friedrich Schneider & Heinrich W. Ursprung, 2008. "The 2008 GEA Journal‐Ranking for the Economics Profession," German Economic Review, Verein für Socialpolitik, vol. 9(4), pages 532-538, November.
    16. Frances Ruane & Richard S. J. Tol, 2008. "Rational (successive) h-indices: An application to economics in the Republic of Ireland," Scientometrics, Springer;Akadémiai Kiadó, vol. 75(2), pages 395-405, May.
    17. Frances Ruane & Richard S.J. Tol, 2007. "Refined (Successive) H-Indices: An Application To Economics In The Republic Of Ireland," Working Papers FNU-130, Research unit Sustainability and Global Change, Hamburg University, revised Mar 2007.
    18. Anne‐Wil Harzing & Ron van der Wal, 2009. "A Google Scholar h‐index for journals: An alternative metric to measure journal impact in economics and business," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(1), pages 41-46, January.
    19. Vladimir Pislyakov, 2009. "Comparing two “thermometers”: Impact factors of 20 leading economic journals according to Journal Citation Reports and Scopus," Scientometrics, Springer;Akadémiai Kiadó, vol. 79(3), pages 541-550, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Marcel Clermont & Alexander Dirksen & Barbara Scheidt & Dirk Tunger, 2017. "Citation metrics as an additional indicator for evaluating research performance? An analysis of their correlations and validity," Business Research, Springer;German Academic Association for Business Research, vol. 10(2), pages 249-279, October.
    2. Alexander Butz & Klaus Wohlrabe, 2016. "Comments and Criticism of the 2015 Economists Rankings of Handelsblatt, the FAZ and RePEc," ifo Schnelldienst, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, vol. 69(10), pages 37-44, May.
    3. Alexander Butz & Klaus Wohlrabe, 2016. "Die Ökonomen-Rankings 2015 von Handelsblatt, FAZ und RePEc: Methodik, Ergebnisse, Kritik und Vergleich," ifo Working Paper Series 212, ifo Institute - Leibniz Institute for Economic Research at the University of Munich.
    4. Andrea Diem & Stefan C. Wolter, 2011. "The Use of Bibliometrics to Measure Research Performance in Education Sciences," Economics of Education Working Paper Series 0066, University of Zurich, Department of Business Administration (IBW), revised May 2013.
    5. Klaus Wohlrabe, 2011. "Handelsblatt and RePEc rankings in comparison," ifo Schnelldienst, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, vol. 64(17), pages 66-71, September.
    6. Klaus Wohlrabe, 2013. "A Few Comments on the Handelsblatt Ranking 2013," ifo Schnelldienst, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, vol. 66(23), pages 79-83, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Seiler, Christian & Wohlrabe, Klaus, 2012. "Ranking economists on the basis of many indicators: An alternative approach using RePEc data," Journal of Informetrics, Elsevier, vol. 6(3), pages 389-402.
    2. David L. Anderson & John Tressler, 2017. "Researcher rank stability across alternative output measurement schemes in the context of a time limited research evaluation: the New Zealand case," Applied Economics, Taylor & Francis Journals, vol. 49(45), pages 4542-4553, September.
    3. Wohlrabe, Klaus, 2016. "Taking the Temperature: A Meta-Ranking of Economics Journals," MPRA Paper 68933, University Library of Munich, Germany.
    4. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    5. Matthias Krapf, 2010. "Research evaluation and journal quality weights: Much ado about nothing?," Working Paper Series of the Department of Economics, University of Konstanz 2010-02, Department of Economics, University of Konstanz.
    6. Lutz Bornmann & Alexander Butz & Klaus Wohlrabe, 2018. "What are the top five journals in economics? A new meta-ranking," Applied Economics, Taylor & Francis Journals, vol. 50(6), pages 659-675, February.
    7. Müller, Harry, 2012. "Die Zitationshäufigkeit als Qualitätsindikator im Rahmen der Forschungsleistungsmessung," Discussion Papers of the Institute for Organisational Economics 1/2012, University of Münster, Institute for Organisational Economics.
    8. David L. Anderson & John Tressler, 2013. "The Relevance of the “h-” and “g-” Index to Economics in the Context of A Nation-Wide Research Evaluation Scheme: The New Zealand Case," Economic Papers, The Economic Society of Australia, vol. 32(1), pages 81-94, March.
    9. Zhenbin Yan & Qiang Wu & Xingchen Li, 2016. "Do Hirsch-type indices behave the same in assessing single publications? An empirical study of 29 bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1815-1833, December.
    10. Klaus Wohlrabe, 2011. "Handelsblatt and RePEc rankings in comparison," ifo Schnelldienst, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, vol. 64(17), pages 66-71, September.
    11. Christian Zimmermann, 2013. "Academic Rankings with RePEc," Econometrics, MDPI, vol. 1(3), pages 1-32, December.
    12. Albers, Sönke, 2008. "Three Failed Attempts of Joint Rankings of Research in Economics and Business," MPRA Paper 12868, University Library of Munich, Germany, revised 16 Jan 2009.
    13. Damien Besancenot & Abdelghani Maddi, 2019. "Should citations be weighted to assess the influence of an academic article?," Economics Bulletin, AccessEcon, vol. 435(1), pages 435-445.
    14. Joseph Gerald Hirschberg & Jeanette Ngaire Lye, 2020. "Grading Journals In Economics: The Abcs Of The Abdc," Journal of Economic Surveys, Wiley Blackwell, vol. 34(4), pages 876-921, September.
    15. Jan-Egbert Sturm & Heinrich W. Ursprung, 2017. "The Handelsblatt Rankings 2.0: Research Rankings for the Economics Profession in Austria, Germany, and Switzerland," German Economic Review, Verein für Socialpolitik, vol. 18(4), pages 492-515, November.
    16. Giovanni Anania & Annarosa Caruso, 2013. "Two simple new bibliometric indexes to better evaluate research in disciplines where publications typically receive less citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(2), pages 617-631, August.
    17. Krapf, Matthias & Ursprung, Heinrich W. & Zimmermann, Christian, 2017. "Parenthood and productivity of highly skilled labor: Evidence from the groves of academe," Journal of Economic Behavior & Organization, Elsevier, vol. 140(C), pages 147-175.
    18. Mingers, John & Leydesdorff, Loet, 2015. "A review of theory and practice in scientometrics," European Journal of Operational Research, Elsevier, vol. 246(1), pages 1-19.
    19. Glenn Ellison, 2013. "How Does the Market Use Citation Data? The Hirsch Index in Economics," American Economic Journal: Applied Economics, American Economic Association, vol. 5(3), pages 63-90, July.
    20. George Emm Halkos & Nickolaos G. Tzeremes, 2011. "Measuring economic journals’ citation efficiency: a data envelopment analysis approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(3), pages 979-1001, September.

    More about this item

    JEL classification:

    • L86 - Industrial Organization - - Industry Studies: Services - - - Information and Internet Services; Computer Software
    • Z00 - Other Special Topics - - General - - - General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ces:ifosdt:v:63:y:2010:i:21:p:18-28. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Klaus Wohlrabe (email available below). General contact details of provider: https://edirc.repec.org/data/ifooode.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.