IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v129y2024i2d10.1007_s11192-023-04911-2.html
   My bibliography  Save this article

Is there a “difference-in-difference”? The impact of scientometric evaluation on the evolution of international publications in Egyptian universities and research centres

Author

Listed:
  • Mona Farouk Ali

    (Helwan University)

Abstract

Adopting scientometric evaluation in the context of academic promotion has been one of the recent reforms in Egyptian higher education geared towards boosting scientific excellence and global exposure. This type of evaluation awards higher points to works published in Web of Science journals. The objective here is to investigate the hypothesis that this procedure leads to the intensification of international publications and shifts in publication patterns in Egyptian universities compared to research centres that are not subject to the regulations employing this type of evaluation. The requisite research data were extracted from Clarivate's InCites for 45 universities and 43 research centres from 1996 to 2021 and analysed using the difference-in-difference approach. The findings show a generally positive effect on WoS publications. That is, significant differences are observed for all variables examined for universities (total publications, subjects, types, and journal quartiles), whereas this evaluation had some spillover effects on research centres. This proves the significance of the evaluation in fostering international productivity. The outputs would be of interest to policy-makers who are planning to promote and broaden the scope of this novel technique in research assessment. The present study may also stimulate future comparisons with other countries applying this type of evaluation.

Suggested Citation

  • Mona Farouk Ali, 2024. "Is there a “difference-in-difference”? The impact of scientometric evaluation on the evolution of international publications in Egyptian universities and research centres," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(2), pages 1119-1154, February.
    • Handle: RePEc:spr:scient:v:129:y:2024:i:2:d:10.1007_s11192-023-04911-2
    DOI: 10.1007/s11192-023-04911-2
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-023-04911-2
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-023-04911-2?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ju-fang Shao & Hui-yun Shen, 2012. "Research assessment and monetary rewards: the overemphasized impact factor in China," Research Evaluation, Oxford University Press, vol. 21(3), pages 199-203, June.
    2. Aksnes, Dag W. & Rip, Arie, 2009. "Researchers' perceptions of citations," Research Policy, Elsevier, vol. 38(6), pages 895-905, July.
    3. Matveeva, Nataliya & Sterligov, Ivan & Yudkevich, Maria, 2021. "The effect of Russian University Excellence Initiative on publications and collaboration patterns," Journal of Informetrics, Elsevier, vol. 15(1).
    4. Bahaa Ibrahim, 2018. "Arab Spring’s effect on scientific productivity and research performance in Arab countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(3), pages 1555-1586, December.
    5. Korytkowski, Przemyslaw & Kulczycki, Emanuel, 2019. "Publication counting methods for a national research evaluation exercise," Journal of Informetrics, Elsevier, vol. 13(3), pages 804-816.
    6. Björn Hammarfelt & Gaby Haddow, 2018. "Conflicting measures and values: How humanities scholars in Australia and Sweden use and react to bibliometric indicators," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 69(7), pages 924-935, July.
    7. Andrea Giovanni Nuzzolese & Paolo Ciancarini & Aldo Gangemi & Silvio Peroni & Francesco Poggi & Valentina Presutti, 2019. "Do altmetrics work for assessing research quality?," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(2), pages 539-562, February.
    8. Nataliya Matveeva & Anuška Ferligoj, 2020. "Scientific collaboration in Russian universities before and after the excellence initiative Project 5-100," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(3), pages 2383-2407, September.
    9. Jamal El-Ouahi & Vincent Larivière, 2023. "On the lack of women researchers in the Middle East and North Africa," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(8), pages 4321-4348, August.
    10. Hamdi A. Al-Jamimi & Galal M. BinMakhashen & Lutz Bornmann, 2022. "Use of bibliometrics for research evaluation in emerging markets economies: a review and discussion of bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(10), pages 5879-5930, October.
    11. Przemysław Korytkowski & Emanuel Kulczycki, 2019. "Examining how country-level science policy shapes publication patterns: the case of Poland," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(3), pages 1519-1543, June.
    12. Daniella Bayle Deutz & Thea Marie Drachen & Dorte Drongstrup & Niels Opstrup & Charlotte Wien, 2021. "Quantitative quality: a study on how performance-based measures may change the publication patterns of Danish researchers," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(4), pages 3303-3320, April.
    13. Roula Inglesi-Lotz & Anastassios Pouris, 2011. "Scientometric impact assessment of a research policy instrument: the case of rating researchers on scientific outputs in South Africa," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(3), pages 747-760, September.
    14. Björn Hammarfelt & Sarah de Rijcke, 2015. "Accountability in context: effects of research evaluation systems on publication practices, disciplinary norms, and individual working routines in the faculty of Arts at Uppsala University," Research Evaluation, Oxford University Press, vol. 24(1), pages 63-77.
    15. Woodside, Arch G., 2009. "Journal and author impact metrics: An editorial," Journal of Business Research, Elsevier, vol. 62(1), pages 1-4, January.
    16. Hamdi A. Al-Jamimi & Galal M. BinMakhashen & Lutz Bornmann & Yousif Ahmed Al Wajih, 2023. "Saudi Arabia research: academic insights and trend analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(10), pages 5595-5627, October.
    17. Peter Ingwersen & Birger Larsen, 2014. "Influence of a performance indicator on Danish research production and citation impact 2000–12," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1325-1344, November.
    18. Andrei V. Grinëv & Daria S. Bylieva & Victoria V. Lobatyuk, 2021. "Russian University Teachers’ Perceptions of Scientometrics," Publications, MDPI, vol. 9(2), pages 1-16, May.
    19. Sarah de Rijcke & Paul F. Wouters & Alex D. Rushforth & Thomas P. Franssen & Björn Hammarfelt, 2016. "Evaluation practices and effects of indicator use—a literature review," Research Evaluation, Oxford University Press, vol. 25(2), pages 161-169.
    20. Camil Demetrescu & Irene Finocchi & Andrea Ribichini & Marco Schaerf, 2020. "On bibliometrics in academic promotions: a case study in computer science and engineering in Italy," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(3), pages 2207-2228, September.
    21. Marzolla, Moreno, 2016. "Assessing evaluation procedures for individual researchers: The case of the Italian National Scientific Qualification," Journal of Informetrics, Elsevier, vol. 10(2), pages 408-438.
    22. Cruz-Castro, Laura & Sanz-Menendez, Luis, 2021. "What should be rewarded? Gender and evaluation criteria for tenure and promotion," Journal of Informetrics, Elsevier, vol. 15(3).
    23. Emanuel Kulczycki, 2019. "Field patterns of scientometric indicators use for presenting research portfolio for assessment," Research Evaluation, Oxford University Press, vol. 28(2), pages 169-181.
    24. Diana Hicks & Paul Wouters & Ludo Waltman & Sarah de Rijcke & Ismael Rafols, 2015. "Bibliometrics: The Leiden Manifesto for research metrics," Nature, Nature, vol. 520(7548), pages 429-431, April.
    25. Demir, Selcuk Besir, 2018. "Predatory journals: Who publishes in them and why?," Journal of Informetrics, Elsevier, vol. 12(4), pages 1296-1311.
    26. Alec P. Christie & David Abecasis & Mehdi Adjeroud & Juan C. Alonso & Tatsuya Amano & Alvaro Anton & Barry P. Baldigo & Rafael Barrientos & Jake E. Bicknell & Deborah A. Buhl & Just Cebrian & Ricardo , 2020. "Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences," Nature Communications, Nature, vol. 11(1), pages 1-11, December.
    27. Giovanni Abramo & Ciriaco Andrea D’Angelo, 2021. "The different responses of universities to introduction of performance-based research funding [How Incentives Trickle down: Local Use of a National Bibliometric Indicator System]," Research Evaluation, Oxford University Press, vol. 30(4), pages 514-528.
    28. Giovanni Abramo & Ciriaco Andrea D’Angelo & Myroslava Hladchenko, 2023. "Assessing the effects of publication requirements for professorship on research performance and publishing behaviour of Ukrainian academics," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(8), pages 4589-4609, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mona Farouk Ali, 2025. "Investigating shifts in publication patterns after launching scientometric evaluation at Egyptian universities: an analysis of submitted research for promotion," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(3), pages 1751-1787, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mona Farouk Ali, 2025. "Investigating shifts in publication patterns after launching scientometric evaluation at Egyptian universities: an analysis of submitted research for promotion," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(3), pages 1751-1787, March.
    2. Leonid Gokhberg & Tatiana Kuznetsova & Maxim Kotsemir, 2023. "From the Soviet Union to the Russian Federation: publication activity dynamics along the evolution of national science policies," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(11), pages 6195-6246, November.
    3. Ramón A. Feenstra & Emilio Delgado López-Cózar, 2022. "Philosophers’ appraisals of bibliometric indicators and their use in evaluation: from recognition to knee-jerk rejection," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(4), pages 2085-2103, April.
    4. Eugenio Petrovich, 2022. "Bibliometrics in Press. Representations and uses of bibliometric indicators in the Italian daily newspapers," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2195-2233, May.
    5. Giovanni Abramo & Ciriaco Andrea D’Angelo & Myroslava Hladchenko, 2023. "Assessing the effects of publication requirements for professorship on research performance and publishing behaviour of Ukrainian academics," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(8), pages 4589-4609, August.
    6. Renata Kudaibergenova & Sandugash Uzakbay & Asselya Makanova & Kymbat Ramadinkyzy & Erlan Kistaubayev & Ruslan Dussekeev & Kadyrzhan Smagulov, 2022. "Managing publication change at Al-Farabi Kazakh National University: a case study," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(1), pages 453-479, January.
    7. Dag W. Aksnes & Liv Langfeldt & Paul Wouters, 2019. "Citations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories," SAGE Open, , vol. 9(1), pages 21582440198, February.
    8. Seeber, Marco & Cattaneo, Mattia & Meoli, Michele & Malighetti, Paolo, 2019. "Self-citations as strategic response to the use of metrics for career decisions," Research Policy, Elsevier, vol. 48(2), pages 478-491.
    9. Yaşar Tonta & Müge Akbulut, 2020. "Does monetary support increase citation impact of scholarly papers?," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1617-1641, November.
    10. Lutz Bornmann & Julian N. Marewski, 2019. "Heuristics as conceptual lens for understanding and studying the usage of bibliometrics in research evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(2), pages 419-459, August.
    11. Mehmet Pinar, 2023. "Do research performances of universities and disciplines in England converge or diverge? An assessment of the progress between research excellence frameworks in 2014 and 2021," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(10), pages 5731-5766, October.
    12. Kaile Gong, 2023. "Twenty years of Chinese social sciences towards internationalization (1998–2017): a knowledge sources perspective," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(12), pages 6373-6402, December.
    13. Emanuel Kulczycki & Ying Huang & Alesia A. Zuccala & Tim C. E. Engels & Antonio Ferrara & Raf Guns & Janne Pölönen & Gunnar Sivertsen & Zehra Taşkın & Lin Zhang, 2022. "Uses of the Journal Impact Factor in national journal rankings in China and Europe," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(12), pages 1741-1754, December.
    14. Pantea Kamrani & Isabelle Dorsch & Wolfgang G. Stock, 2021. "Do researchers know what the h-index is? And how do they estimate its importance?," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 5489-5508, July.
    15. Wildgaard, Lorna, 2016. "A critical cluster analysis of 44 indicators of author-level performance," Journal of Informetrics, Elsevier, vol. 10(4), pages 1055-1078.
    16. Moxin Li & Yang Wang, 2024. "Influence of political tensions on scientific productivity, citation impact, and knowledge combinations," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(4), pages 2337-2370, April.
    17. Katerina Guba, 2024. "Why do sociologists on academic periphery willingly support bibliometric indicators?," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(1), pages 497-518, January.
    18. Milojević, Staša & Radicchi, Filippo & Bar-Ilan, Judit, 2017. "Citation success index − An intuitive pair-wise journal comparison metric," Journal of Informetrics, Elsevier, vol. 11(1), pages 223-231.
    19. Tony Ross-Hellauer & Thomas Klebel & Petr Knoth & Nancy Pontika, 2024. "Value dissonance in research(er) assessment: individual and perceived institutional priorities in review, promotion, and tenure," Science and Public Policy, Oxford University Press, vol. 51(3), pages 337-351.
    20. Andrey Lovakov & Jaime A. Teixeira da Silva, 2025. "Scientometric indicators in research evaluation and research misconduct: analysis of the Russian university excellence initiative," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(3), pages 1813-1829, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:129:y:2024:i:2:d:10.1007_s11192-023-04911-2. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.