IDEAS home Printed from https://ideas.repec.org/a/bla/econpa/v32y2013i1p81-94.html
   My bibliography  Save this article

The Relevance of the “h-” and “g-” Index to Economics in the Context of A Nation-Wide Research Evaluation Scheme: The New Zealand Case

Author

Listed:
  • David L. Anderson
  • John Tressler

Abstract

The purpose of this paper is to explore the relevance of the citation-based ‘h’ and ‘g’ indexes as a means for measuring research output in economics. This study is unique in that it is the first to utilize the ‘h’ and ‘g’ indexes in the context of a time limited evaluation period and to provide comprehensive coverage of all academic economists in all university-based economics departments within a nation state. For illustration purposes we have selected New Zealand’s Performance Based Research Fund (PBRF) as our evaluation scheme. In order to provide a frame of reference for ‘h’ and ‘g’ index output measures, we have also estimated research output using a number of journal-based weighting schemes. In general, our findings suggest that ‘h’ and ‘g’ index scores are strongly associated with low-powered journal ranking schemes and weakly associated with high powered journal weighting schemes. More specifically, we found the ‘h’ and ‘g’ indexes to suffer from a lack of differentiation: for example, 52 percent of all participants received a score of zero under both measures, and 92 and 89 percent received scores of two or less under ‘h’ and ‘g’, respectively. Overall, our findings suggest that ‘h’ and ‘g’ indexes should not be incorporated into a PBRF-like framework.
(This abstract was borrowed from another version of this item.)

Suggested Citation

  • David L. Anderson & John Tressler, 2013. "The Relevance of the “h-” and “g-” Index to Economics in the Context of A Nation-Wide Research Evaluation Scheme: The New Zealand Case," Economic Papers, The Economic Society of Australia, vol. 32(1), pages 81-94, March.
    • Handle: RePEc:bla:econpa:v:32:y:2013:i:1:p:81-94
    DOI: 10.1111/1759-3441.12026
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1111/10.1111/1759-3441.12026
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1111/1759-3441.12026?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Pantelis Kalaitzidakis & Theofanis P. Mamuneas & Thanasis Stengos, 2011. "An updated ranking of academic journals in economics," Canadian Journal of Economics, Canadian Economics Association, vol. 44(4), pages 1525-1538, November.
    2. Kodrzycki Yolanda K. & Yu Pingkang, 2006. "New Approaches to Ranking Economics Journals," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 5(1), pages 1-44, August.
    3. Leo Egghe, 2006. "Theory and practise of the g-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 69(1), pages 131-152, October.
    4. Tol, Richard S.J., 2008. "A rational, successive g-index applied to economics departments in Ireland," Journal of Informetrics, Elsevier, vol. 2(2), pages 149-155.
    5. Richard S. J. Tol, 2009. "The h-index and its alternatives: An application to the 100 most prolific economists," Scientometrics, Springer;Akadémiai Kiadó, vol. 80(2), pages 317-324, August.
    6. Magnus Henrekson & Daniel Waldenström, 2011. "How Should Research Performance Be Measured? A Study Of Swedish Economists," Manchester School, University of Manchester, vol. 79(6), pages 1139-1156, December.
    7. Ignacio Palacios-Huerta & Oscar Volij, 2004. "The Measurement of Intellectual Influence," Econometrica, Econometric Society, vol. 72(3), pages 963-977, May.
    8. Kalaitzidakis, P. & Mamuneas, T.P. & Stengos, T., 2003. "Rankings of Academic Journals and Institutions," Working Papers 2003-8, University of Guelph, Department of Economics and Finance.
    9. Glenn Ellison, 2013. "How Does the Market Use Citation Data? The Hirsch Index in Economics," American Economic Journal: Applied Economics, American Economic Association, vol. 5(3), pages 63-90, July.
    10. Andrew J. Oswald, 2007. "An Examination of the Reliability of Prestigious Scholarly Journals: Evidence and Implications for Decision‐Makers," Economica, London School of Economics and Political Science, vol. 74(293), pages 21-31, February.
    11. Harris, G T, 1990. "Research Output in Australian University Economics Departments: An Update for 1984-88," Australian Economic Papers, Wiley Blackwell, vol. 29(55), pages 249-259, December.
    12. Chia‐Lin Chang & Michael McAleer & Les Oxley, 2011. "What Makes A Great Journal Great In Economics? The Singer Not The Song," Journal of Economic Surveys, Wiley Blackwell, vol. 25(2), pages 326-361, April.
    13. Liebowitz, S J & Palmer, J P, 1984. "Assessing the Relative Impacts of Economic Journals," Journal of Economic Literature, American Economic Association, vol. 22(1), pages 77-88, March.
    14. Pantelis Kalaitzidakis & Theofanis P. Mamuneas & Thanasis Stengos, 2003. "Rankings of Academic Journals and Institutions in Economics," Journal of the European Economic Association, MIT Press, vol. 1(6), pages 1346-1366, December.
    15. Jeong-Yoo Kim & Insik Min & Christian Zimmermann, 2011. "The Economics of Citation," Korean Economic Review, Korean Economic Association, vol. 27, pages 93-114.
    16. Courtault, Jean-Michel & Hayek, Naïla & Rimbaux, Eric & Zhu, Tong, 2010. "Research in economics and management in France: A bibliometric study using the h-index," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 39(2), pages 329-337, April.
    17. Richard Pomfret & Liang Choon Wang, 2003. "Evaluating The Research Output Of Australian Universities' Economics Departments," Australian Economic Papers, Wiley Blackwell, vol. 42(4), pages 418-441, December.
    18. William H. Starbuck, 2005. "How Much Better Are the Most-Prestigious Journals? The Statistics of Academic Publication," Organization Science, INFORMS, vol. 16(2), pages 180-200, April.
    19. Joseph Macri & Dipendra Sinha, 2006. "Rankings Methodology for International Comparisons of Institutions and Individuals: an Application to Economics in Australia and New Zealand," Journal of Economic Surveys, Wiley Blackwell, vol. 20(1), pages 111-156, February.
    20. Andrew R. Bodman, 2010. "Measuring the influentialness of economic geographers during the 'great half century': an approach using the h index," Journal of Economic Geography, Oxford University Press, vol. 10(1), pages 141-156, January.
    21. Frances Ruane & Richard S. J. Tol, 2008. "Rational (successive) h-indices: An application to economics in the Republic of Ireland," Scientometrics, Springer;Akadémiai Kiadó, vol. 75(2), pages 395-405, May.
    22. Martin Ravallion & Adam Wagstaff, 2011. "On measuring scholarly influence by citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(1), pages 321-337, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. David L. Anderson & John Tressler, 2013. "The New Zealand performance-based research fund and its impact on publication activity in economics," Research Evaluation, Oxford University Press, vol. 23(1), pages 1-11, September.
    2. Wai Ching Poon & Gareth D. Leeves, 2017. "Research output: evidence from economics departments in the Asia-Pacific region," Journal of the Asia Pacific Economy, Taylor & Francis Journals, vol. 22(4), pages 604-620, October.
    3. David L. Anderson & John Tressler, 2014. "Citation-Capture Rates by Economic Journals:Do they Differ from Other Disciplines and Does it Matter?," Working Papers in Economics 14/10, University of Waikato.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. David L. Anderson & John Tressler, 2017. "Researcher rank stability across alternative output measurement schemes in the context of a time limited research evaluation: the New Zealand case," Applied Economics, Taylor & Francis Journals, vol. 49(45), pages 4542-4553, September.
    2. Seiler, Christian & Wohlrabe, Klaus, 2012. "Ranking economists on the basis of many indicators: An alternative approach using RePEc data," Journal of Informetrics, Elsevier, vol. 6(3), pages 389-402.
    3. John Gibson & David L. Anderson & John Tressler, 2014. "Which Journal Rankings Best Explain Academic Salaries? Evidence From The University Of California," Economic Inquiry, Western Economic Association International, vol. 52(4), pages 1322-1340, October.
    4. Wohlrabe, Klaus, 2016. "Taking the Temperature: A Meta-Ranking of Economics Journals," MPRA Paper 68933, University Library of Munich, Germany.
    5. David L. Anderson & John Tressler, 2011. "The Merits of Using Citations to Measure Research Output in Economics Departments: The New Zealand Case," Working Papers in Economics 11/11, University of Waikato.
    6. David Anderson & John Tressler, 2008. "Research output in New Zealand economics departments 2000-2006: A stock approach," New Zealand Economic Papers, Taylor & Francis Journals, vol. 42(2), pages 155-189.
    7. Lutz Bornmann & Alexander Butz & Klaus Wohlrabe, 2018. "What are the top five journals in economics? A new meta-ranking," Applied Economics, Taylor & Francis Journals, vol. 50(6), pages 659-675, February.
    8. David L. Anderson & John Tressler, 2010. "The Merits of Using Citation-Based Journal Weighting Schemes to Measure Research Performance in Economics: The Case of New Zealand," Working Papers in Economics 10/03, University of Waikato.
    9. David L. Anderson & John Tressler, 2009. "The Excellence in Research for Australia Scheme: An Evaluation of the Draft Journal Weights for Economics," Working Papers in Economics 09/07, University of Waikato.
    10. Bräuninger, Michael & Haucap, Justus & Muck, Johannes, 2011. "Was lesen und schätzen Ökonomen im Jahr 2011?," DICE Ordnungspolitische Perspektiven 18, Heinrich Heine University Düsseldorf, Düsseldorf Institute for Competition Economics (DICE).
    11. John Gibson & David L. Anderson & John Tressler, 2017. "Citations Or Journal Quality: Which Is Rewarded More In The Academic Labor Market?," Economic Inquiry, Western Economic Association International, vol. 55(4), pages 1945-1965, October.
    12. John Tressler & David L. Anderson, 2012. "Citations as a Measure of the Research Outputs of New Zealand's Economics Departments: The Problem of 'Long and Variable Lags'," Agenda - A Journal of Policy Analysis and Reform, Australian National University, College of Business and Economics, School of Economics, vol. 19(1), pages 17-40.
    13. Stelios Katranidis & Theodore Panagiotidis & Costas Zontanos, 2014. "An Evaluation Of The Greek Universities’ Economics Departments," Bulletin of Economic Research, Wiley Blackwell, vol. 66(2), pages 173-182, April.
    14. Bräuninger Michael & Haucap Justus & Muck Johannes, 2011. "Was lesen und schätzen deutschsprachige Ökonomen heute?," Perspektiven der Wirtschaftspolitik, De Gruyter, vol. 12(4), pages 339-371, November.
    15. J. Atsu Amegashie, 2020. "Citations And Incentives In Academic Contests," Economic Inquiry, Western Economic Association International, vol. 58(3), pages 1233-1244, July.
    16. David L. Anderson & John Tressler, 2009. "The ‘Excellence in Research for Australia’ Scheme: A Test Drive of Draft Journal Weights with New Zealand Data," Agenda - A Journal of Policy Analysis and Reform, Australian National University, College of Business and Economics, School of Economics, vol. 16(4), pages 7-24.
    17. Johannes König & David I. Stern & Richard S.J. Tol, 2022. "Confidence Intervals for Recursive Journal Impact Factors," Tinbergen Institute Discussion Papers 22-038/III, Tinbergen Institute.
    18. David L. Anderson & John Tressler, 2008. "Research Output in New Zealand Economics Department 2000-2006," Working Papers in Economics 08/05, University of Waikato.
    19. Ham, John C. & Wright, Julian & Ye, Ziqiu, 2023. "Documenting and Explaining the Dramatic Rise of the New Society Journals in Economics," IZA Discussion Papers 16337, Institute of Labor Economics (IZA).
    20. George Emm Halkos & Nickolaos G. Tzeremes, 2011. "Measuring economic journals’ citation efficiency: a data envelopment analysis approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(3), pages 979-1001, September.

    More about this item

    JEL classification:

    • A19 - General Economics and Teaching - - General Economics - - - Other
    • C81 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Methodology for Collecting, Estimating, and Organizing Microeconomic Data; Data Access
    • J24 - Labor and Demographic Economics - - Demand and Supply of Labor - - - Human Capital; Skills; Occupational Choice; Labor Productivity

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:econpa:v:32:y:2013:i:1:p:81-94. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/esausea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.