IDEAS home Printed from https://ideas.repec.org/p/wai/econwp/11-11.html
   My bibliography  Save this paper

The Merits of Using Citations to Measure Research Output in Economics Departments: The New Zealand Case

Author

Listed:

Abstract

In this paper we explore the merits of utilizing citation counts to measure research output in economics in the context of a nation-wide research evaluation scheme. We selected one such system for study: the New Zealand government’s Programme-Based Research Fund (PBRF). Citations were collected for all refereed papers produced by New Zealand’s academic economists over the period 2000 to 2008 using the databases of the ISI/Web of Science and, to a limited extent, Google Scholar. These data allowed us to estimate the time lags in economics between publication of an article and the flow of citations; to demonstrate the impact of alternative definitions of ‘economics-relevant’ journals on citation counts; and to assess the impact of direct citation measures and alternative schemes on departmental and individual performance. Our findings suggest that the time-lags between publication and citing are such that it would be difficult to rely on citations counts to produce a meaningful measure of output in a PBRF-like research evaluation framework, especially one based explicitly on individual assessment.

Suggested Citation

  • David L. Anderson & John Tressler, 2011. "The Merits of Using Citations to Measure Research Output in Economics Departments: The New Zealand Case," Working Papers in Economics 11/11, University of Waikato.
  • Handle: RePEc:wai:econwp:11/11
    as

    Download full text from publisher

    File URL: ftp://wms-webprod1.mngt.waikato.ac.nz/RePEc/wai/econwp/1111.pdf
    Download Restriction: no

    References listed on IDEAS

    as
    1. Pantelis Kalaitzidakis & Theofanis P. Mamuneas & Thanasis Stengos, 2011. "An updated ranking of academic journals in economics," Canadian Journal of Economics, Canadian Economics Association, vol. 44(4), pages 1525-1538, November.
    2. David L. Anderson & John Tressler, 2010. "The Merits of Using Citation-Based Journal Weighting Schemes to Measure Research Performance in Economics: The Case of New Zealand," Working Papers in Economics 10/03, University of Waikato.
    3. Wall Howard J, 2009. "Don't Get Skewed Over by Journal Rankings," The B.E. Journal of Economic Analysis & Policy, De Gruyter, pages 1-12.
    4. Magnus Henrekson & Daniel Waldenström, 2011. "How Should Research Performance Be Measured? A Study Of Swedish Economists," Manchester School, University of Manchester, pages 1139-1156.
    5. Andrew J. Oswald, 2007. "An Examination of the Reliability of Prestigious Scholarly Journals: Evidence and Implications for Decision-Makers," Economica, London School of Economics and Political Science, pages 21-31.
    6. Laband, David N & Piette, Michael J, 1994. "The Relative Impacts of Economics Journals: 1970-1990," Journal of Economic Literature, American Economic Association, pages 640-666.
    7. Liebowitz, S J & Palmer, J P, 1984. "Assessing the Relative Impacts of Economic Journals," Journal of Economic Literature, American Economic Association, pages 77-88.
    8. Pedro Cosme Vieira & Aurora A.C. Teixeira, 2006. "Are Finance, Management, and Marketing Autonomous Fields of Scientific Research? An Analysis Based on Journal Citations," FEP Working Papers 233, Universidade do Porto, Faculdade de Economia do Porto.
    9. Joseph Macri & Dipendra Sinha, 2006. "Rankings Methodology for International Comparisons of Institutions and Individuals: an Application to Economics in Australia and New Zealand," Journal of Economic Surveys, Wiley Blackwell, vol. 20(1), pages 111-156, February.
    10. Tom Coupé, 2003. "Revealed Performances: Worldwide Rankings of Economists and Economics Departments, 1990-2000," Journal of the European Economic Association, MIT Press, vol. 1(6), pages 1309-1345, December.
    Full references (including those not matched with items on IDEAS)

    More about this item

    Keywords

    citations; economics departments; journal weighting schemes; PBRF; research output;

    JEL classification:

    • A19 - General Economics and Teaching - - General Economics - - - Other
    • C81 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Methodology for Collecting, Estimating, and Organizing Microeconomic Data; Data Access
    • J24 - Labor and Demographic Economics - - Demand and Supply of Labor - - - Human Capital; Skills; Occupational Choice; Labor Productivity

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wai:econwp:11/11. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Brian Silverstone). General contact details of provider: http://edirc.repec.org/data/dewaknz.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.