IDEAS home Printed from https://ideas.repec.org/p/een/crwfrp/1501.html

Research Assessment Using Early Citation Information

Author

Listed:
  • Stephan B. Bruns

    (University of Kassel - Meta-Research in Economics Group)

  • David I. Stern

    (Crawford School of Public Policy, The Australian National University)

Abstract

We show that university rankings in economics based on long-run citation counts can be easily predicted using early citations. The rank correlation between universitiesÕ cumulative citations received over ten years for economics articles published in 2003 and 2004 and citations received in 2003 to 2004 alone is 0.91 in the UK and 0.82 in Australia. We compare these citation-based university rankings with the rankings of the 2008 Research Assessment Exercise in the UK and the 2010 Excellence in Research assessment in Australia. Rank correlations are quite strong but there are differences between rankings based on this type of peer review and rankings based on citation counts. However, if assessors are willing to consider citation analysis to assess some disciplines as is the case for the natural sciences and psychology in Australia there seems no reason to not include economics in this set.

Suggested Citation

  • Stephan B. Bruns & David I. Stern, 2015. "Research Assessment Using Early Citation Information," Crawford School Research Papers 1501, Crawford School of Public Policy, The Australian National University.
    • Handle: RePEc:een:crwfrp:1501
    as

    Download full text from publisher

    File URL: http://ssrn.com/abstract=2602151
    Download Restriction: no
    ---><---

    Other versions of this item:

    Citations

    Blog mentions

    As found by EconAcademics.org, the blog aggregator for Economics research:
    1. My Submission to Stern Review of the REF
      by noreply@blogger.com (David Stern) in Stochastic Trend on 2016-03-03 09:34:00
    2. Mid-Year Update
      by noreply@blogger.com (David Stern) in Stochastic Trend on 2016-06-01 08:40:00
    3. Annual Review 2016
      by noreply@blogger.com (David Stern) in Stochastic Trend on 2016-12-26 17:08:00

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. David L. Anderson & John Tressler, 2017. "Researcher rank stability across alternative output measurement schemes in the context of a time limited research evaluation: the New Zealand case," Applied Economics, Taylor & Francis Journals, vol. 49(45), pages 4542-4553, September.
    2. David L. Anderson & John Tressler, 2016. "Citation-Capture Rates for Economics Journals: Do they Differ from Other Disciplines and Does it Matter?," Economic Papers, The Economic Society of Australia, vol. 35(1), pages 73-85, March.
    3. Thelwall, Mike & Nevill, Tamara, 2018. "Could scientists use Altmetric.com scores to predict longer term citation counts?," Journal of Informetrics, Elsevier, vol. 12(1), pages 237-248.
    4. Kousha, Kayvan & Thelwall, Mike & Abdoli, Mahshid, 2018. "Can Microsoft Academic assess the early citation impact of in-press articles? A multi-discipline exploratory analysis," Journal of Informetrics, Elsevier, vol. 12(1), pages 287-298.
    5. Wan Siti Nur Aiza & Liyana Shuib & Norisma Idris & Nur Baiti Afini Normadhi, 2024. "Features, techniques and evaluation in predicting articles’ citations: a review from years 2010–2023," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(1), pages 1-29, January.
    6. David Anderson & John Tressler, 2018. "The Impact of Citation Timing: A Framework and Examples," Review of Economics and Institutions, Università di Perugia, vol. 9(2).
    7. John Gibson & David L. Anderson & John Tressler, 2017. "Citations Or Journal Quality: Which Is Rewarded More In The Academic Labor Market?," Economic Inquiry, Western Economic Association International, vol. 55(4), pages 1945-1965, October.
    8. Andrea Fronzetti Colladon & Ciriaco Andrea D’Angelo & Peter A. Gloor, 2020. "Predicting the future success of scientific publications through social network and semantic analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(1), pages 357-377, July.
    9. Syed Hasan & Robert Breunig, 2021. "Article length and citation outcomes," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 7583-7608, September.
    10. Vasilios D. Kosteas, 2018. "Predicting long-run citation counts for articles in top economics journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(3), pages 1395-1412, June.
    11. Basso, Antonella & di Tollo, Giacomo, 2022. "Prediction of UK research excellence framework assessment by the departmental h-index," European Journal of Operational Research, Elsevier, vol. 296(3), pages 1036-1049.
    12. David L. Anderson & John Tressler, 2015. "Are Researcher Rankings Stable Across Alternative Output Measurement Schemes in the Context of a Time Limited Research Evaluation? The New Zealand Case," Working Papers in Economics 15/10, University of Waikato.
    13. Abramo, Giovanni & D’Angelo, Ciriaco Andrea & Felici, Giovanni, 2019. "Predicting publication long-term impact through a combination of early citations and journal impact factor," Journal of Informetrics, Elsevier, vol. 13(1), pages 32-49.
    14. Björn Dressel & David I. Stern, 2021. "Research at public policy schools in the Asia‐Pacific region ranked," Asia and the Pacific Policy Studies, Wiley Blackwell, vol. 8(1), pages 151-166, January.

    More about this item

    JEL classification:

    • A14 - General Economics and Teaching - - General Economics - - - Sociology of Economics

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:een:crwfrp:1501. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: David Stern (email available below). General contact details of provider: https://edirc.repec.org/data/asanuau.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.