IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v84y2010i1d10.1007_s11192-009-0087-x.html
   My bibliography  Save this article

A suggested method for the measurement of world-leading research (illustrated with data on economics)

Author

Listed:
  • Andrew J. Oswald

    (University of Warwick)

Abstract

Countries often spend billions on university research. There is growing interest in how to assess whether that money is well spent. Is there an objective way to assess the quality of a nation’s world-leading science? I suggest a method, and illustrate it with data on economics. Of 450 genuinely world-leading journal articles, the UK produced 10% and the rest of Europe slightly more. Interestingly, more than a quarter of these UK articles came from outside the best-known university departments. The proposed methodology could be applied to almost any academic discipline or nation.

Suggested Citation

  • Andrew J. Oswald, 2010. "A suggested method for the measurement of world-leading research (illustrated with data on economics)," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(1), pages 99-113, July.
    • Handle: RePEc:spr:scient:v:84:y:2010:i:1:d:10.1007_s11192-009-0087-x
    DOI: 10.1007/s11192-009-0087-x
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-009-0087-x
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-009-0087-x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. DREZE, Jacques & ESTEVAN, Fernanda, 2006. "Research and higher education in economics: can we deliver the Lisbon objectives ?," LIDAM Discussion Papers CORE 2006051, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    2. Ana Rute Cardoso & Paulo Guimarães & Klaus F. Zimmermann, 2010. "Comparing the early research performance of PhD graduates in labor economics in Europe and the USA," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(3), pages 621-637, September.
    3. Brazier, John & Roberts, Jennifer & Deverill, Mark, 2002. "The estimation of a preference-based measure of health from the SF-36," Journal of Health Economics, Elsevier, vol. 21(2), pages 271-292, March.
    4. Andrew J. Oswald, 2007. "An Examination of the Reliability of Prestigious Scholarly Journals: Evidence and Implications for Decision‐Makers," Economica, London School of Economics and Political Science, vol. 74(293), pages 21-31, February.
    5. Bruce A. Weinberg, 2009. "An Assessment of British Science over the Twentieth Century," Economic Journal, Royal Economic Society, vol. 119(538), pages 252-269, June.
    6. Im, Kyung So & Pesaran, M. Hashem & Shin, Yongcheol, 2003. "Testing for unit roots in heterogeneous panels," Journal of Econometrics, Elsevier, vol. 115(1), pages 53-74, July.
    7. Amanda H Goodall, 2005. "Should Research Universities be Led by Top Researchers? Part 1: Are they?," HEW 0506003, University Library of Munich, Germany.
    8. J. Peter Neary & James A. Mirrlees & Jean Tirole, 2003. "Evaluating Economics Research in Europe: An Introduction," Journal of the European Economic Association, MIT Press, vol. 1(6), pages 1239-1249, December.
    9. Hamermesh, Daniel S. & Pfann, Gerard A., 2009. "Markets for Reputation: Evidence on Quality and Quantity in Academe," IZA Discussion Papers 4610, Institute of Labor Economics (IZA).
    10. Daniel S. Hamermesh & Peter Schmidt, 2003. "The Determinants of Econometric Society Fellows Elections," Econometrica, Econometric Society, vol. 71(1), pages 399-407, January.
    11. Frey, Bruno S, 2003. "Publishing as Prostitution?--Choosing between One's Own Ideas and Academic Success," Public Choice, Springer, vol. 116(1-2), pages 205-223, July.
    12. William H. Starbuck, 2005. "How Much Better Are the Most-Prestigious Journals? The Statistics of Academic Publication," Organization Science, INFORMS, vol. 16(2), pages 180-200, April.
    13. Jonathan Adams, 2005. "Early citation counts correlate with accumulated impact," Scientometrics, Springer;Akadémiai Kiadó, vol. 63(3), pages 567-581, June.
    14. Glenn Ellison, 2011. "Is Peer Review In Decline?," Economic Inquiry, Western Economic Association International, vol. 49(3), pages 635-657, July.
    15. John Hudson, 2007. "Be known by the company you keep: Citations — quality or chance?," Scientometrics, Springer;Akadémiai Kiadó, vol. 71(2), pages 231-238, May.
    16. Machin, Stephen & Oswald, Andrew, 2000. "UK Economics and the Future Supply of Academic Economists," Economic Journal, Royal Economic Society, vol. 110(464), pages 334-349, June.
    17. Stephen Wu, 2007. "Recent publishing trends at the AER, JPE and QJE," Applied Economics Letters, Taylor & Francis Journals, vol. 14(1), pages 59-63.
    18. Scott Smart & Joel Waldfogel, 1996. "A Citation-Based Test for Discrimination at Economics and Finance Journals," NBER Working Papers 5460, National Bureau of Economic Research, Inc.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Rose, Michael E. & Opolot, Daniel C. & Georg, Co-Pierre, 2022. "Discussants," Research Policy, Elsevier, vol. 51(10).
    2. Daniel Sgroi & Andrew J. Oswald, 2013. "How Should Peer‐review Panels Behave?," Economic Journal, Royal Economic Society, vol. 0, pages 255-278, August.
    3. Michael C. Calver & Maggie Lilith & Christopher R. Dickman, 2013. "A ‘perverse incentive’ from bibliometrics: could National Research Assessment Exercises (NRAEs) restrict literature availability for nature conservation?," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(1), pages 243-255, April.
    4. Oswald, Andrew J., 2015. "The Objective Measurement of World-Leading Research," IZA Discussion Papers 8829, Institute of Labor Economics (IZA).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Oswald, Andrew J., 2009. "World-Leading Research and its Measurement," Economic Research Papers 271312, University of Warwick - Department of Economics.
    2. Oswald, Andrew J., 2015. "The Objective Measurement of World-Leading Research," IZA Discussion Papers 8829, Institute of Labor Economics (IZA).
    3. Oswald, Andrew J., 2008. "Can We Test for Bias in Scientific Peer-Review?," IZA Discussion Papers 3665, Institute of Labor Economics (IZA).
    4. Daniel S. Hamermesh, 2018. "Citations in Economics: Measurement, Uses, and Impacts," Journal of Economic Literature, American Economic Association, vol. 56(1), pages 115-156, March.
    5. RosalindS. Hunter & Andrew J. Oswald & Bruce G. Charlton, 2009. "The Elite Brain Drain," Economic Journal, Royal Economic Society, vol. 119(538), pages 231-251, June.
    6. Ali, Showkat & Carden, Giles & Culling, Benjamin & Hunter, Rosalind & Oswald, Andrew J & Owen, Nicola & Ralsmark, Hilda & Snodgrass, Natalie, 2007. "Elite Scientists and the Global Brain Drain," The Warwick Economics Research Paper Series (TWERPS) 825, University of Warwick, Department of Economics.
    7. Rosalind S Hunter, 2009. "The Elite Brain Drain," Working Papers id:2048, eSocialSciences.
    8. Jinyoung Kim & Kanghyock Koh, 2014. "Incentives for Journal Editors," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 47(1), pages 348-371, February.
    9. Jan Ours & Frederic Vermeulen, 2007. "Ranking Dutch Economists," De Economist, Springer, vol. 155(4), pages 469-487, December.
    10. KRAPF, Matthias & SCHLÄPFER, Jörg, 2012. "How Nobel Laureates Would Perform In The Handelsblatt Ranking," Regional and Sectoral Economic Studies, Euro-American Association of Economic Development, vol. 12(3).
    11. Leks Borgans & Frank Corvers (Transl. by: E. Pokatovich ), 2010. "The americanization of European higher education and research," Voprosy obrazovaniya / Educational Studies Moscow, National Research University Higher School of Economics, issue 2, pages 5-43.
    12. Rolf Ketzler & Klaus F. Zimmermann, 2013. "A citation-analysis of economic research institutes," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(3), pages 1095-1112, June.
    13. Jens Prüfer & David Zetland, 2010. "An auction market for journal articles," Public Choice, Springer, vol. 145(3), pages 379-403, December.
    14. Marcella Corsi & Carlo D'Ippoliti & Federico Lucidi, 2010. "Pluralism at Risk? Heterodox Economic Approaches and the Evaluation of Economic Research in Italy," American Journal of Economics and Sociology, Wiley Blackwell, vol. 69(5), pages 1495-1529, November.
    15. Berlemann, Michael & Haucap, Justus, 2015. "Which factors drive the decision to opt out of individual research rankings? An empirical study of academic resistance to change," Research Policy, Elsevier, vol. 44(5), pages 1108-1115.
    16. Berlemann, Michael & Haucap, Justus, 2012. "Which factors drive the decision to boycott and opt out of research rankings? A note," DICE Discussion Papers 72, Heinrich Heine University Düsseldorf, Düsseldorf Institute for Competition Economics (DICE).
    17. Osterloh, Margit & Frey, Bruno S., 2020. "How to avoid borrowed plumes in academia," Research Policy, Elsevier, vol. 49(1).
    18. Michael Berlemann & Justus Haucap, 2012. "Which Factors Drive the Decision to Boycott and Opt Out of Research Rankings?," CESifo Working Paper Series 3997, CESifo.
    19. Matteo Migheli & Giovanni B. Ramello, 2014. "Open Access Journals & Academics’ Behaviour," ICER Working Papers 03-2014, ICER - International Centre for Economic Research.
    20. David L. Anderson & John Tressler, 2013. "The Relevance of the “h-” and “g-” Index to Economics in the Context of A Nation-Wide Research Evaluation Scheme: The New Zealand Case," Economic Papers, The Economic Society of Australia, vol. 32(1), pages 81-94, March.

    More about this item

    Keywords

    Science; Evaluation; European economics; United Kingdom; Peer-review; Research Excellence Framework (REF); Citations; Research assessment exercise (RAE);
    All these keywords.

    JEL classification:

    • A1 - General Economics and Teaching - - General Economics
    • O38 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Government Policy

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:84:y:2010:i:1:d:10.1007_s11192-009-0087-x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.