IDEAS home Printed from https://ideas.repec.org/p/usi/wpaper/614.html
   My bibliography  Save this paper

Evaluating Research Activity:Impact Factor vs. Research Factor

Author

Listed:
  • Silvia Ferrini
  • Marco P. Tucci

Abstract

The Impact Factor (IF) “has moved ... from an obscure bibliometric indicator to become the chief quantitative measure of the quality of a journal, its research papers, the researchers who wrote those papers, and even the institution they work in” ([2], p. 1). However, the use of this index for evaluating individual scientists is dubious. The present work compares the ranking of research units generated by the Research Factor (RF) index with that associated with the popular IF. The former, originally introduced in [38], reflects article and book publications and a host of other activities categorized as coordination activities (e.g., conference organization, research group coordination), dissemination activities (e.g., conference and seminar presentations, participation in research group), editorial activities (e.g., journal editor, associate editor, referee) and functional activities (e.g., Head of Department). The main conclusion is that by replacing the IF with the RF in hiring, tenure decisions and awarding of grants would greatly increase the number of topics investigated and the number and quality of long run projects.

Suggested Citation

  • Silvia Ferrini & Marco P. Tucci, 2011. "Evaluating Research Activity:Impact Factor vs. Research Factor," Department of Economics University of Siena 614, Department of Economics, University of Siena.
    • Handle: RePEc:usi:wpaper:614
    as

    Download full text from publisher

    File URL: http://repec.deps.unisi.it/quaderni/614.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Martin Davies & Joe Hirschberg & Jenny Lye & Carol Johnston & Ian Mcdonald, 2007. "Systematic Influences On Teaching Evaluations: The Case For Caution," Australian Economic Papers, Wiley Blackwell, vol. 46(1), pages 18-38, March.
    2. Lokman I. Meho & Kiduk Yang, 2007. "Impact of data sources on citation counts and rankings of LIS faculty: Web of science versus scopus and google scholar," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 58(13), pages 2105-2125, November.
    3. Andrew J. Oswald, 2007. "An Examination of the Reliability of Prestigious Scholarly Journals: Evidence and Implications for Decision‐Makers," Economica, London School of Economics and Political Science, vol. 74(293), pages 21-31, February.
    4. Phillips, Peter C.B., 1985. "Professor J. D. Sargan," Econometric Theory, Cambridge University Press, vol. 1(01), pages 119-139, April.
    5. Cooley, Thomas F & Prescott, Edward C, 1976. "Estimation in the Presence of Stochastic Parameter Variation," Econometrica, Econometric Society, vol. 44(1), pages 167-184, January.
    6. Michael Bräuninger & Justus Haucap, 2003. "Reputation and Relevance of Economics Journals," Kyklos, Wiley Blackwell, vol. 56(2), pages 175-197, May.
    7. Stigler, George J & Stigler, Stephen M & Friedland, Claire, 1995. "The Journals of Economics," Journal of Political Economy, University of Chicago Press, vol. 103(2), pages 331-359, April.
    8. Thed N. van Leeuwen & Henk F. Moed, 2002. "Development and application of journal impact measures in the Dutch science system," Scientometrics, Springer;Akadémiai Kiadó, vol. 53(2), pages 249-266, February.
    9. Tom Coupé, 2003. "Revealed Performances: Worldwide Rankings of Economists and Economics Departments, 1990-2000," Journal of the European Economic Association, MIT Press, vol. 1(6), pages 1309-1345, December.
    10. Marco p. Tucci & Sandra Fontani & Silvia Ferrini, 2008. "L’ “R-Factor”: un nuovo modo di valutare la ricerca scientifica," Department of Economics University of Siena 527, Department of Economics, University of Siena.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Justus Haucap & Johannes Muck, 2015. "What drives the relevance and reputation of economics journals? An update from a survey among economists," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(3), pages 849-877, June.
    2. John Gibson & David L. Anderson & John Tressler, 2017. "Citations Or Journal Quality: Which Is Rewarded More In The Academic Labor Market?," Economic Inquiry, Western Economic Association International, vol. 55(4), pages 1945-1965, October.
    3. John Tressler & David L. Anderson, 2012. "Citations as a Measure of the Research Outputs of New Zealand's Economics Departments: The Problem of 'Long and Variable Lags'," Agenda - A Journal of Policy Analysis and Reform, Australian National University, College of Business and Economics, School of Economics, vol. 19(1), pages 17-40.
    4. Besancenot, Damien & Vranceanu, Radu, 2008. "Can incentives for research harm research? A business schools' tale," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 37(3), pages 1248-1265, June.
    5. Thiago Dumont Oliveira & Marwil J. Dávila-Fernández, 2020. "From modelmania to datanomics? The rise of mathematical and quantitative methods in three top economics journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(1), pages 51-70, April.
    6. Kocher, Martin G. & Luptacik, Mikulas & Sutter, Matthias, 2006. "Measuring productivity of research in economics: A cross-country study using DEA," Socio-Economic Planning Sciences, Elsevier, vol. 40(4), pages 314-332, December.
    7. Klaus Ritzberger, 2008. "Eine invariante Bewertung wirtschaftswissenschaftlicher Fachzeitschriften," Perspektiven der Wirtschaftspolitik, Verein für Socialpolitik, vol. 9(3), pages 267-285, August.
    8. Mingers, John & Yang, Liying, 2017. "Evaluating journal quality: A review of journal citation indicators and ranking in business and management," European Journal of Operational Research, Elsevier, vol. 257(1), pages 323-337.
    9. Rolf Ketzler & Klaus F. Zimmermann, 2013. "A citation-analysis of economic research institutes," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(3), pages 1095-1112, June.
    10. Damien Besancenot & Abdelghani Maddi, 2019. "Should citations be weighted to assess the influence of an academic article?," Economics Bulletin, AccessEcon, vol. 435(1), pages 435-445.
    11. Bräuninger Michael & Haucap Justus & Muck Johannes, 2011. "Was lesen und schätzen deutschsprachige Ökonomen heute?," Perspektiven der Wirtschaftspolitik, De Gruyter, vol. 12(4), pages 339-371, November.
    12. Wohlrabe, Klaus, 2016. "Taking the Temperature: A Meta-Ranking of Economics Journals," MPRA Paper 68933, University Library of Munich, Germany.
    13. David I. Stern, 2013. "Uncertainty Measures for Economics Journal Impact Factors," Journal of Economic Literature, American Economic Association, vol. 51(1), pages 173-189, March.
    14. Mingers, John & Leydesdorff, Loet, 2015. "A review of theory and practice in scientometrics," European Journal of Operational Research, Elsevier, vol. 246(1), pages 1-19.
    15. Damien Besancenot & Kim Huynh & Joao Faria, 2012. "Search and research: the influence of editorial boards on journals’ quality," Theory and Decision, Springer, vol. 73(4), pages 687-702, October.
    16. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    17. David L. Anderson & John Tressler, 2017. "Researcher rank stability across alternative output measurement schemes in the context of a time limited research evaluation: the New Zealand case," Applied Economics, Taylor & Francis Journals, vol. 49(45), pages 4542-4553, September.
    18. Victoria Anauati & Sebastian Galiani & Ramiro H. Gálvez, 2016. "Quantifying The Life Cycle Of Scholarly Articles Across Fields Of Economic Research," Economic Inquiry, Western Economic Association International, vol. 54(2), pages 1339-1355, April.
    19. Martin Gregor, 2006. "Hodnocení ekonomických pracovišť a ekonomů: Koho, proč, čím a jak [A survey of rankings of economic departments: Global, american, european and national]," Politická ekonomie, Prague University of Economics and Business, vol. 2006(3), pages 394-414.
    20. David L. Anderson & John Tressler, 2011. "The Merits of Using Citations to Measure Research Output in Economics Departments: The New Zealand Case," Working Papers in Economics 11/11, University of Waikato.

    More about this item

    Keywords

    scientific research assessment; Impact Factor; bibliometric indices; feasible Research Factor;
    All these keywords.

    JEL classification:

    • A11 - General Economics and Teaching - - General Economics - - - Role of Economics; Role of Economists
    • A12 - General Economics and Teaching - - General Economics - - - Relation of Economics to Other Disciplines

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:usi:wpaper:614. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Fabrizio Becatti (email available below). General contact details of provider: https://edirc.repec.org/data/desieit.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.