IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v101y2014i1d10.1007_s11192-014-1407-3.html
   My bibliography  Save this article

Science and technology metrics for research policy evaluation: some insights from a Moroccan experience

Author

Listed:
  • Hamid Bouabid

    (Mohammed V - Agdal University)

Abstract

If peer review has been and is continuing to be an acceptable approach for evaluation, Science and technology (S&T) metrics have been demonstrated to be a more accurate and objectively independent tools for evaluation. This article provides insights from an example of a relevant use of S&T metrics to assess a national research policy and subsequently universities achievements within this policy. One of the main findings were that just by setting S&T metrics as objective indicators there was an increasing research outputs: productivity, impact, and collaboration. However, overall productivity is still far low when brought to academic staff size and that a huge difference exists among universities achievements. The reliability of scientometric evaluation’s use as a performance tool is increasing in universities and the culture of this evaluation usefulness in research policy has widely spread. Surprisingly, this evaluation shows that even if S&T metrics have substantially increased, funds execution as means of rate of payment on total budget was less than 15 % due mainly to the high and unusual increase in funding allocations than was before the policy, a fact to which universities were managerially not well prepared. Finally, future evaluation should follow in the very short-term to quantify the impact extent of the policy revealed in this annual evaluation.

Suggested Citation

  • Hamid Bouabid, 2014. "Science and technology metrics for research policy evaluation: some insights from a Moroccan experience," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 899-915, October.
  • Handle: RePEc:spr:scient:v:101:y:2014:i:1:d:10.1007_s11192-014-1407-3
    DOI: 10.1007/s11192-014-1407-3
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-014-1407-3
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s11192-014-1407-3?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Hamid Bouabid & Vincent Larivière, 2013. "The lengthening of papers’ life expectancy: a diachronous analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(3), pages 695-717, December.
    2. Elizabeth S. Vieira & José A. N. F. Gomes, 2009. "A comparison of Scopus and Web of Science for a typical university," Scientometrics, Springer;Akadémiai Kiadó, vol. 81(2), pages 587-600, November.
    3. Giovanni Abramo & Ciriaco Andrea D’Angelo, 2011. "Evaluating research: from informed peer review to bibliometrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(3), pages 499-514, June.
    4. Loet Leydesdorff & Tobias Opthof, 2010. "Scopus's source normalized impact per paper (SNIP) versus a journal impact factor based on fractional counting of citations," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 61(11), pages 2365-2369, November.
    5. Nick Haslam & Lauren Ban & Leah Kaufmann & Stephen Loughnan & Kim Peters & Jennifer Whelan & Sam Wilson, 2008. "What makes an article influential? Predicting impact in social and personality psychology," Scientometrics, Springer;Akadémiai Kiadó, vol. 76(1), pages 169-185, July.
    6. Thomas Scherngell & Yuanjia Hu, 2011. "Collaborative Knowledge Production in China: Regional Evidence from a Gravity Model Approach," Regional Studies, Taylor & Francis Journals, vol. 45(6), pages 755-772.
    7. Éric Archambault & David Campbell & Yves Gingras & Vincent Larivière, 2009. "Comparing bibliometric statistics obtained from the Web of Science and Scopus," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(7), pages 1320-1326, July.
    8. Ed J. Rinia, 2000. "Scientometric Studies and Their Role in Research Policy of Two Research Councils in the Netherlands," Scientometrics, Springer;Akadémiai Kiadó, vol. 47(2), pages 363-378, February.
    9. Hicks, Diana, 2012. "Performance-based university research funding systems," Research Policy, Elsevier, vol. 41(2), pages 251-261.
    10. van Rijnsoever, Frank J. & Hessels, Laurens K. & Vandeberg, Rens L.J., 2008. "A resource-based view on the interactions of university researchers," Research Policy, Elsevier, vol. 37(8), pages 1255-1266, September.
    11. Auranen, Otto & Nieminen, Mika, 2010. "University research funding and publication performance--An international comparison," Research Policy, Elsevier, vol. 39(6), pages 822-834, July.
    12. Defazio, Daniela & Lockett, Andy & Wright, Mike, 2009. "Funding incentives, collaborative dynamics and scientific productivity: Evidence from the EU framework program," Research Policy, Elsevier, vol. 38(2), pages 293-305, March.
    13. Primož Južnič & Stojan Pečlin & Matjaž Žaucer & Tilen Mandelj & Miro Pušnik & Franci Demšar, 2010. "Scientometric indicators: peer-review, bibliometric methods and conflict of interests," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(2), pages 429-441, November.
    14. Henri Delanghe & Brian Sloan & Ugur Muldur, 2011. "European research policy and bibliometric indicators, 1990–2005," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(2), pages 389-398, May.
    15. Frank J. van Rijnsoever & Laurens K. Hessels & Rens L.J. Vandeberg, 2008. "A resource-based view on the interactions of university researchers," Innovation Studies Utrecht (ISU) working paper series 08-14, Utrecht University, Department of Innovation Studies, revised Apr 2008.
    16. Besiki Stvilia & Charles C. Hinnant & Katy Schindler & Adam Worrall & Gary Burnett & Kathleen Burnett & Michelle M. Kazmer & Paul F. Marty, 2011. "Composition of scientific teams and publication productivity at a national science lab," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(2), pages 270-283, February.
    17. Hamid Bouabid & Mohamed Dalimi & Zayer ElMajid, 2011. "Impact evaluation of the voluntary early retirement policy on research and technology outputs of the faculties of science in Morocco," Scientometrics, Springer;Akadémiai Kiadó, vol. 86(1), pages 125-132, January.
    18. Besiki Stvilia & Charles C. Hinnant & Katy Schindler & Adam Worrall & Gary Burnett & Kathleen Burnett & Michelle M. Kazmer & Paul F. Marty, 2011. "Composition of scientific teams and publication productivity at a national science lab," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(2), pages 270-283, February.
    19. L Quoniam & H Rostaing & E Boutin & H Dou, 1995. "Treating bibliometric indicators with caution: their dependence on the source database," Research Evaluation, Oxford University Press, vol. 5(3), pages 177-181, December.
    20. Giovanni Abramo & Ciriaco Andrea D’Angelo & Marco Solazzi, 2011. "The relationship between scientists’ research performance and the degree of internationalization of their research," Scientometrics, Springer;Akadémiai Kiadó, vol. 86(3), pages 629-643, March.
    21. Daniel Torres-Salinas & Emilio Delgado Lopez-Cózar & Evaristo Jiménez-Contreras, 2009. "Ranking of departments and researchers within a university using two different databases: Web of Science versus Scopus," Scientometrics, Springer;Akadémiai Kiadó, vol. 80(3), pages 761-774, September.
    22. Vincent Larivière & Alesia Zuccala & Éric Archambault, 2008. "The declining scientific impact of theses: Implications for electronic thesis and dissertation repositories and graduate studies," Scientometrics, Springer;Akadémiai Kiadó, vol. 74(1), pages 109-121, January.
    23. Nils T. Hagen, 2010. "Deconstructing doctoral dissertations: how many papers does it take to make a PhD?," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(2), pages 567-579, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Andrey Guskov & Denis Kosyakov & Irina Selivanova, 2016. "Scientometric research in Russia: impact of science policy changes," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(1), pages 287-303, April.
    2. Olena Leonchuk & Denis O. Gray, 2019. "Scientific and technological (human) social capital formation and Industry–University Cooperative Research Centers: a quasi-experimental evaluation of graduate student outcomes," The Journal of Technology Transfer, Springer, vol. 44(5), pages 1638-1664, October.
    3. Hamid Bouabid & Hind Achachi, 2022. "Size of science team at university and internal co-publications: science policy implications," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 6993-7013, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Giovanni Abramo & Ciriaco Andrea D’Angelo & Gianluca Murgia, 2014. "Variation in research collaboration patterns across academic ranks," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(3), pages 2275-2294, March.
    2. Ruslan Rakhmatullin & Louis Brennan, 2014. "Motivation Behind Researchers’ Participation in Formal Networking Research Projects Funded by the European Union," Journal of the Knowledge Economy, Springer;Portland International Center for Management of Engineering and Technology (PICMET), vol. 5(2), pages 305-329, June.
    3. Esther Hormiga & Petra Saá-Pérez & Nieves L. Díaz-Díaz & José Luis Ballesteros-Rodríguez & Inmaculada Aguiar-Diaz, 2017. "The influence of entrepreneurial orientation on the performance of academic research groups: the mediating role of knowledge sharing," The Journal of Technology Transfer, Springer, vol. 42(1), pages 10-32, February.
    4. Svein Kyvik & Ingvild Reymert, 2017. "Research collaboration in groups and networks: differences across academic fields," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 951-967, November.
    5. Hamid Bouabid & Hind Achachi, 2022. "Size of science team at university and internal co-publications: science policy implications," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 6993-7013, December.
    6. Edler, Jakob & Fier, Heide & Grimpe, Christoph, 2011. "International scientist mobility and the locus of knowledge and technology transfer," Research Policy, Elsevier, vol. 40(6), pages 791-805, July.
    7. Karacic, Domagoj & Miskulin, Ivan & Serdarusic, Hrvoje, 2016. "State Investment In Science And Scientific Productivity Of Universities," UTMS Journal of Economics, University of Tourism and Management, Skopje, Macedonia, vol. 7(1), pages 37-48.
    8. Rivera-Huerta, René & Dutrénit, Gabriela & Ekboir, Javier Mario & Sampedro, José Luis & Vera-Cruz, Alexandre O., 2011. "Do linkages between farmers and academic researchers influence researcher productivity? The Mexican case," Research Policy, Elsevier, vol. 40(7), pages 932-942, September.
    9. Migheli, Matteo & Zotti, Roberto, 2020. "The strange case of the Matthew effect and beauty contests: Research evaluation and specialisation in Italian universities," Socio-Economic Planning Sciences, Elsevier, vol. 71(C).
    10. Maaike Verbree & Edwin Horlings & Peter Groenewegen & Inge Weijden & Peter Besselaar, 2015. "Organizational factors influencing scholarly performance: a multivariate study of biomedical research groups," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 25-49, January.
    11. Luigi Aldieri & Gennaro Guida & Maxim Kotsemir & Concetto Paolo Vinci, 2019. "An investigation of impact of research collaboration on academic performance in Italy," Quality & Quantity: International Journal of Methodology, Springer, vol. 53(4), pages 2003-2040, July.
    12. Tomaz Bartol & Gordana Budimir & Doris Dekleva-Smrekar & Miro Pusnik & Primoz Juznic, 2014. "Assessment of research fields in Scopus and Web of Science in the view of national research evaluation in Slovenia," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(2), pages 1491-1504, February.
    13. José Álvarez-García & Amador Durán-Sánchez & María de la Cruz Del Río-Rama & Diego Fernando García-Vélez, 2018. "Active Ageing: Mapping of Scientific Coverage," IJERPH, MDPI, vol. 15(12), pages 1-21, December.
    14. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    15. Cimenler, Oguz & Reeves, Kingsley A. & Skvoretz, John, 2014. "A regression analysis of researchers’ social network metrics on their citation performance in a college of engineering," Journal of Informetrics, Elsevier, vol. 8(3), pages 667-682.
    16. Jordi Ardanuy, 2012. "Scientific collaboration in Library and Information Science viewed through the Web of Knowledge: the Spanish case," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(3), pages 877-890, March.
    17. Kasia Zalewska-Kurek & Klaudia Egedova & Peter A. Th. M. Geurts & Hans E. Roosendaal, 2018. "Knowledge transfer activities of scientists in nanotechnology," The Journal of Technology Transfer, Springer, vol. 43(1), pages 139-158, February.
    18. Rivera, Rene & Sampedro, Jose Luis & Dutrenit, Gabriela & Ekboir, Javier Mario & Vera-Cruz, Alexandre O., 2009. "How productive are academic researchers in agriculture-related sciences? The Mexican case," MERIT Working Papers 2009-038, United Nations University - Maastricht Economic and Social Research Institute on Innovation and Technology (MERIT).
    19. Daniele Rotolo & Michael Hopkins & Nicola Grassano, 2023. "Do funding sources complement or substitute? Examining the impact of cancer research publications," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 74(1), pages 50-66, January.
    20. Abramo, Giovanni & D’Angelo, Ciriaco Andrea & Di Costa, Flavia, 2019. "When research assessment exercises leave room for opportunistic behavior by the subjects under evaluation," Journal of Informetrics, Elsevier, vol. 13(3), pages 830-840.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:101:y:2014:i:1:d:10.1007_s11192-014-1407-3. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.