IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v87y2011i3d10.1007_s11192-011-0352-7.html
   My bibliography  Save this article

Evaluating research: from informed peer review to bibliometrics

Author

Listed:
  • Giovanni Abramo

    (National Research Council of Italy
    Laboratory for Studies of Research and Technology Transfer, School of Engineering, University of Rome “Tor Vergata”)

  • Ciriaco Andrea D’Angelo

    (Laboratory for Studies of Research and Technology Transfer, School of Engineering, University of Rome “Tor Vergata”)

Abstract

National research assessment exercises are becoming regular events in ever more countries. The present work contrasts the peer-review and bibliometrics approaches in the conduct of these exercises. The comparison is conducted in terms of the essential parameters of any measurement system: accuracy, robustness, validity, functionality, time and costs. Empirical evidence shows that for the natural and formal sciences, the bibliometric methodology is by far preferable to peer-review. Setting up national databases of publications by individual authors, derived from Web of Science or Scopus databases, would allow much better, cheaper and more frequent national research assessments.

Suggested Citation

  • Giovanni Abramo & Ciriaco Andrea D’Angelo, 2011. "Evaluating research: from informed peer review to bibliometrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(3), pages 499-514, June.
    • Handle: RePEc:spr:scient:v:87:y:2011:i:3:d:10.1007_s11192-011-0352-7
    DOI: 10.1007/s11192-011-0352-7
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-011-0352-7
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-011-0352-7?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Saul Lach & Mark Schankerman, 2008. "Incentives and invention in universities," RAND Journal of Economics, RAND Corporation, vol. 39(2), pages 403-433, June.
    2. Dag W. Aksnes, 2008. "When different persons have an identical author name. How frequent are homonyms?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(5), pages 838-841, March.
    3. repec:adr:anecst:y:1998:i:49-50:p:05 is not listed on IDEAS
    4. James D. Adams & Zvi Griliches, 1998. "Research Productivity in a System of Universities," Annals of Economics and Statistics, GENES, issue 49-50, pages 127-162.
    5. Giovanni Abramo & Ciriaco Andrea D’Angelo & Fabio Pugini, 2008. "The measurement of Italian universities’ research productivity by a non parametric-bibliometric methodology," Scientometrics, Springer;Akadémiai Kiadó, vol. 76(2), pages 225-244, August.
    6. Giovanni Abramo & Ciriaco Andrea D’Angelo, 2011. "National-scale research performance assessment at the individual level," Scientometrics, Springer;Akadémiai Kiadó, vol. 86(2), pages 347-364, February.
    7. Giovanni Abramo & Ciriaco Andrea D’Angelo & Fulvio Viel, 2010. "Peer review research assessment: a sensitivity analysis of performance rankings to the share of research product evaluated," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(3), pages 705-720, December.
    8. Rinia, E. J. & van Leeuwen, Th. N. & van Vuren, H. G. & van Raan, A. F. J., 1998. "Comparative analysis of a set of bibliometric indicators and central peer review criteria: Evaluation of condensed matter physics in the Netherlands," Research Policy, Elsevier, vol. 27(1), pages 95-107, May.
    9. Abramo, Giovanni & D'Angelo, Ciriaco Andrea & Caprasecca, Alessandro, 2009. "Allocative efficiency in public research funding: Can bibliometrics help?," Research Policy, Elsevier, vol. 38(1), pages 206-215, February.
    10. Saul Lach & Mark Schankerman, 2003. "Incentives and invention in universities," Proceedings, Federal Reserve Bank of San Francisco, issue nov.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Abramo, Giovanni & Cicero, Tindaro & D’Angelo, Ciriaco Andrea, 2011. "A field-standardized application of DEA to national-scale research assessment of universities," Journal of Informetrics, Elsevier, vol. 5(4), pages 618-628.
    2. Giovanni Abramo & Ciriaco Andrea D’Angelo & Flavia Di Costa, 2011. "National research assessment exercises: the effects of changing the rules of the game during the game," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(1), pages 229-238, July.
    3. Lee Branstetter & Kwon Hyeog Ug, 2004. "The Restructuring Of Japanese Research And Development: The Increasing Impact Of Science On Japanese R&D," Discussion papers 04021, Research Institute of Economy, Trade and Industry (RIETI).
    4. Giovanni Abramo & Ciriaco Andrea D’Angelo & Flavia Di Costa, 2011. "National research assessment exercises: a comparison of peer review and bibliometrics rankings," Scientometrics, Springer;Akadémiai Kiadó, vol. 89(3), pages 929-941, December.
    5. Giovanni Abramo & Ciriaco Andrea D’Angelo & Marco Solazzi, 2010. "National research assessment exercises: a measure of the distortion of performance rankings when labor input is treated as uniform," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(3), pages 605-619, September.
    6. Jean-Paul Chavas & Bradford Barham & Jeremy Foltz & Kwansoo Kim, 2012. "Analysis and decomposition of scope economies: R&D at US research universities," Applied Economics, Taylor & Francis Journals, vol. 44(11), pages 1387-1404, April.
    7. Banal-Estanol, A. & Macho-Stadler, I., 2007. "Financial incentives in academia: research versus development," Working Papers 07/09, Department of Economics, City St George's, University of London.
    8. Balconi, Margherita & Laboranti, Andrea, 2006. "University-industry interactions in applied research: The case of microelectronics," Research Policy, Elsevier, vol. 35(10), pages 1616-1630, December.
    9. Macho-Stadler, Ines & Perez-Castrillo, David & Veugelers, Reinhilde, 2007. "Licensing of university inventions: The role of a technology transfer office," International Journal of Industrial Organization, Elsevier, vol. 25(3), pages 483-510, June.
    10. Schankerman, Mark & Belenzon, Sharon, 2007. "Harnessing Success: Determinants of University Technology Licensing Performance," CEPR Discussion Papers 6120, C.E.P.R. Discussion Papers.
    11. Calderini, Mario & Franzoni, Chiara & Vezzulli, Andrea, 2007. "If star scientists do not patent: The effect of productivity, basicness and impact on the decision to patent in the academic world," Research Policy, Elsevier, vol. 36(3), pages 303-319, April.
    12. Abramo, Giovanni & Cicero, Tindaro & D’Angelo, Ciriaco Andrea, 2012. "A sensitivity analysis of researchers’ productivity rankings to the time of citation observation," Journal of Informetrics, Elsevier, vol. 6(2), pages 192-201.
    13. Janet Bercovitz & Maryann Feldman, 2008. "Academic Entrepreneurs: Organizational Change at the Individual Level," Organization Science, INFORMS, vol. 19(1), pages 69-89, February.
    14. Franceschet, Massimo & Costantini, Antonio, 2011. "The first Italian research assessment exercise: A bibliometric perspective," Journal of Informetrics, Elsevier, vol. 5(2), pages 275-291.
    15. Vieira, Elizabeth S. & Cabral, José A.S. & Gomes, José A.N.F., 2014. "How good is a model based on bibliometric indicators in predicting the final decisions made by peers?," Journal of Informetrics, Elsevier, vol. 8(2), pages 390-405.
    16. Li, Jiang & Sanderson, Mark & Willett, Peter & Norris, Michael & Oppenheim, Charles, 2010. "Ranking of library and information science researchers: Comparison of data sources for correlating citation data, and expert judgments," Journal of Informetrics, Elsevier, vol. 4(4), pages 554-563.
    17. Giovanni Abramo & Ciriaco Andrea D'Angelo, 2015. "The VQR, Italy's second national research assessment: Methodological failures and ranking distortions," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(11), pages 2202-2214, November.
    18. Colyvas, Jeannette A., 2007. "From divergent meanings to common practices: The early institutionalization of technology transfer in the life sciences at Stanford University," Research Policy, Elsevier, vol. 36(4), pages 456-476, May.
    19. Lissoni, Francesco, 2010. "Academic inventors as brokers," Research Policy, Elsevier, vol. 39(7), pages 843-857, September.
    20. Nicola Lacetera, 2003. "Incentives and spillovers in R&D activities: an agency-theoretic analysis of industry-university relations," Microeconomics 0312004, University Library of Munich, Germany.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:87:y:2011:i:3:d:10.1007_s11192-011-0352-7. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.