IDEAS home Printed from https://ideas.repec.org/p/sus/susewp/0118.html
   My bibliography  Save this paper

How to Count Citations If You Must: Comment

Author

Listed:
  • David I. Stern

    (Crawford School of Public Policy, The Australian National University)

  • Richard S.J. Tol

    (Department of Economics, University of Sussex
    Department of Spatial Economics, Vrije Universiteit, Amsterdam
    Institute for Environmental Studies, Vrije Universiteit, Amsterdam
    Tinbergen Institute, Amsterdam)

Abstract

Perry and Reny (2016) show that the Euclidean length of a citation list satisfies five axioms including "depth relevance". We explore "breadth relevance", which favors consistent achievers over one-hit wonders. A convex combination of depth and breadth relevant citation metrics does not satisfy the independence axiom, but violations are rare. We estimate the parameters of this metric using two datasets and three rankings, controlling for cohort effects. We find that simply counting citations--neither breadth nor depth--maximizes the correlation between citation index and department rank. However, depth may explain the allocation of researchers across lower ranked departments.

Suggested Citation

  • David I. Stern & Richard S.J. Tol, 2018. "How to Count Citations If You Must: Comment," Working Paper Series 0118, Department of Economics, University of Sussex Business School.
    • Handle: RePEc:sus:susewp:0118
    as

    Download full text from publisher

    File URL: https://www.sussex.ac.uk/webteam/gateway/file.php?name=wps-01-2018.pdf&site=24
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Robert M. Solow, 1956. "A Contribution to the Theory of Economic Growth," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 70(1), pages 65-94.
    2. Nattavudh Powdthavee & Yohanes E. Riyanto & Jack L. Knetsch, 2017. "Impact of Lower Rated Journals on Economists' Judgments of Publication Lists: Evidence from a Survey Experiment," PIER Discussion Papers 63, Puey Ungphakorn Institute for Economic Research.
    3. Glenn Ellison, 2013. "How Does the Market Use Citation Data? The Hirsch Index in Economics," American Economic Journal: Applied Economics, American Economic Association, vol. 5(3), pages 63-90, July.
    4. Tol, Richard S.J., 2013. "The Matthew effect for cohorts of economists," Journal of Informetrics, Elsevier, vol. 7(2), pages 522-527.
    5. Quesada, Antonio, 2011. "Axiomatics for the Hirsch index and the Egghe index," Journal of Informetrics, Elsevier, vol. 5(3), pages 476-480.
    6. Woeginger, Gerhard J., 2008. "An axiomatic characterization of the Hirsch-index," Mathematical Social Sciences, Elsevier, vol. 56(2), pages 224-232, September.
    7. Vladimir G. Deineko & Gerhard J. Woeginger, 2009. "A new family of scientific impact measures: The generalized Kosmulski-indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 80(3), pages 819-826, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Blog mentions

    As found by EconAcademics.org, the blog aggregator for Economics research:
    1. How to Count Citations If You Must
      by noreply@blogger.com (David Stern) in Stochastic Trend on 2018-01-06 05:23:00
    2. Annual Review 2018
      by noreply@blogger.com (David Stern) in Stochastic Trend on 2018-12-23 02:35:00

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. David I. Stern & Richard S. J. Tol, 2021. "Depth and breadth relevance in citation metrics," Economic Inquiry, Western Economic Association International, vol. 59(3), pages 961-977, July.
    2. Bouyssou, Denis & Marchant, Thierry, 2014. "An axiomatic approach to bibliometric rankings and indices," Journal of Informetrics, Elsevier, vol. 8(3), pages 449-477.
    3. Chambers, Christopher P. & Miller, Alan D., 2014. "Scholarly influence," Journal of Economic Theory, Elsevier, vol. 151(C), pages 571-583.
    4. Conan Mukherjee & Ranojoy Basu & Aftab Alam, 2020. "A measure of authorship by publications," Managerial and Decision Economics, John Wiley & Sons, Ltd., vol. 41(3), pages 354-361, April.
    5. Brandão, Luana Carneiro & Soares de Mello, João Carlos Correia Baptista, 2019. "A multi-criteria approach to the h-index," European Journal of Operational Research, Elsevier, vol. 276(1), pages 357-363.
    6. Ana Paula dos Santos Rubem & Ariane Lima Moura & João Carlos Correia Baptista Soares de Mello, 2015. "Comparative analysis of some individual bibliometric indices when applied to groups of researchers," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 1019-1035, January.
    7. Woeginger, Gerhard J., 2014. "Investigations on the step-based research indices of Chambers and Miller," Journal of Informetrics, Elsevier, vol. 8(3), pages 659-666.
    8. Bouyssou, Denis & Marchant, Thierry, 2016. "Ranking authors using fractional counting of citations: An axiomatic approach," Journal of Informetrics, Elsevier, vol. 10(1), pages 183-199.
    9. Osório, António (António Miguel), 2019. "The value and credits of n-authors publications," Working Papers 2072/376026, Universitat Rovira i Virgili, Department of Economics.
    10. Zhenbin Yan & Qiang Wu & Xingchen Li, 2016. "Do Hirsch-type indices behave the same in assessing single publications? An empirical study of 29 bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1815-1833, December.
    11. Mukherjee, Conan & Alam, Aftab, 2016. "On Evaluating Author's Performance by Publications: An Axiomatic Study," Working Papers 2016:14, Lund University, Department of Economics, revised 12 May 2017.
    12. Csató, László, 2019. "Journal ranking should depend on the level of aggregation," Journal of Informetrics, Elsevier, vol. 13(4).
    13. Carlo D'Ippoliti, 2021. "“Many‐Citedness”: Citations Measure More Than Just Scientific Quality," Journal of Economic Surveys, Wiley Blackwell, vol. 35(5), pages 1271-1301, December.
    14. Motty Perry & Philip J. Reny, 2016. "How to Count Citations If You Must," American Economic Review, American Economic Association, vol. 106(9), pages 2722-2741, September.
    15. J. Atsu Amegashie, 2020. "Citations And Incentives In Academic Contests," Economic Inquiry, Western Economic Association International, vol. 58(3), pages 1233-1244, July.
    16. Sylvain Béal & Eric Rémila & Philippe Solal & Sylvain Ferrières, 2016. "An axiomatization of the iterated h-index and applications to sport rankings," Working Papers hal-01394818, HAL.
    17. Haley, M. Ryan & McGee, M. Kevin, 2020. "Jointly valuing journal visibility and author citation count: An axiomatic approach," Journal of Informetrics, Elsevier, vol. 14(1).
    18. Richard S.J. Tol, 2013. "Measuring catch-up growth in malnourished populations," Working Paper Series 6013, Department of Economics, University of Sussex Business School.
    19. Tol, Richard S.J., 2013. "Identifying excellent researchers: A new approach," Journal of Informetrics, Elsevier, vol. 7(4), pages 803-810.
    20. Palacios-Huerta, Ignacio & Volij, Oscar, 2014. "Axiomatic measures of intellectual influence," International Journal of Industrial Organization, Elsevier, vol. 34(C), pages 85-90.

    More about this item

    Keywords

    research assessment; citations;

    JEL classification:

    • A14 - General Economics and Teaching - - General Economics - - - Sociology of Economics
    • C43 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - Index Numbers and Aggregation

    NEP fields

    This paper has been announced in the following NEP Reports:

    Lists

    This item is featured on the following reading lists, Wikipedia, or ReplicationWiki pages:
    1. Papers using RePEc data

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sus:susewp:0118. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: University of Sussex Business School Communications Team (email available below). General contact details of provider: https://edirc.repec.org/data/ecsusuk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.