IDEAS home Printed from https://ideas.repec.org/a/nos/voprob/2020i1p97-125.html
   My bibliography  Save this article

Fictitious Efficiency: What the Russian Survey of Performance of Higher Education Institutions Actually Assessed

Author

Listed:
  • Katerina Guba
  • Mikhail Sokolov
  • Angelika Tsivinskaya

Abstract

Mikhail Sokolov - Candidate of Sciences in Sociology, Professor. E-mail: msokolov@eu.spb.ruKaterina Guba - Candidate of Sciences in Sociology, Director of the Center for Institutional Analysisof Science Education. E-mail: kguba@eu.spb.ruAngelika Tsivinskaya - junior researcher at the Center for Institutional Analysis of Science Education. E-mail: atsivinskaya@eu.spb.ruEuropean University at St. Petersburg. Address: 6/1a Gagarinskaya Str., 191187 St. Petersburg, Russian Federation.Annual Survey of Performance of Higher Education Institutions, conducted in Russia since 2012, was used to identify inefficient universities, subject to measures including closure or merging with other schools. As a result of these policies, the number of Russian universities has decreased more than 1.5 times since 2013. In this article, we analyze the consequences of implementing the appraisal system invented by the Russian Ministry. We argue that the use of the Survey reflects a conceptual confusion between effectiveness (the organization's ability to achieve socially significant goals) and efficiency (the ability to achieve goals with minimal cost). The Ministry has made managerial decisions based on the Survey results (like merging a public university with an allegedly better performing HEI), which indicates understanding of the Survey as an assessment of efficiency. At the same time, statistical analysis of the performance indicators demonstrates that structural characteristics of universities (region and belonging to an institutionalized category, e. g. a pedagogical or an agrarian university) explain the significant part of the variance in the university's performance. This led to discrimination against certain unlucky categories of universities. The methods used include logistic regression to estimate the odds of being labeled as an effective organization in 2014 and the Cox proportional hazards model to estimate the university's chances of survival between 2013 and 2017.

Suggested Citation

  • Katerina Guba & Mikhail Sokolov & Angelika Tsivinskaya, 2020. "Fictitious Efficiency: What the Russian Survey of Performance of Higher Education Institutions Actually Assessed," Voprosy obrazovaniya / Educational Studies Moscow, National Research University Higher School of Economics, issue 1, pages 97-125.
    • Handle: RePEc:nos:voprob:2020:i:1:p:97-125
    as

    Download full text from publisher

    File URL: https://vo.hse.ru/data/2020/04/09/1554550184/Guba.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Brett A. Powell & Diane Suitt Gilleland & L. Carolyn Pearson, 2012. "Expenditures, Efficiency, and Effectiveness in U.S. Undergraduate Higher Education: A National Benchmark Model," The Journal of Higher Education, Taylor & Francis Journals, vol. 83(1), pages 102-127, January.
    2. Mattia Cattaneo & Michele Meoli & Andrea Signori, 2016. "Performance-based funding and university research productivity: the moderating effect of university legitimacy," The Journal of Technology Transfer, Springer, vol. 41(1), pages 85-104, February.
    3. Calabrese, Armando & Capece, Guendalina & Costa, Roberta & Di Pillo, Francesca & Giuffrida, Stefania, 2018. "A ‘power law’ based method to reduce size-related bias in indicators of knowledge performance: An application to university research assessment," Journal of Informetrics, Elsevier, vol. 12(4), pages 1263-1281.
    4. Dag Fjeld Edvardsen & Finn R. Førsund & Sverre A. C. Kittelsen, 2017. "Productivity development of Norwegian institutions of higher education 2004–2013," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 68(4), pages 399-415, April.
    5. Tommaso Agasisti & Ekaterina Shibanova & Daria Platonova & Mikhail Lisyutkin, 2018. "The Russian Excellence Initiative For Higher Education: An Econometric Evaluation Of Short-Term Results," HSE Working papers WP BRP 201/EC/2018, National Research University Higher School of Economics.
    6. Kim S. Cameron, 1986. "Effectiveness as Paradox: Consensus and Conflict in Conceptions of Organizational Effectiveness," Management Science, INFORMS, vol. 32(5), pages 539-553, May.
    7. Mikhail Sokolov, 2017. "The Myth of University Strategy. Market Niches and Organizational Careers of Russian Universities," Voprosy obrazovaniya / Educational Studies Moscow, National Research University Higher School of Economics, issue 2, pages 36-73.
    8. Susanne Warning, 2004. "Performance Differences in German Higher Education: Empirical Analysis of Strategic Groups," Review of Industrial Organization, Springer;The Industrial Organization Society, vol. 24(4), pages 393-408, June.
    9. Zhang, Han & Patton, Donald & Kenney, Martin, 2013. "Building global-class universities: Assessing the impact of the 985 Project," Research Policy, Elsevier, vol. 42(3), pages 765-775.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Губа К. С. & Соколов М. М. & Цивинская А. О., 2020. "Фиктивная Эффективность: Что На Самом Деле Оценивал Мониторинг Эффективности Образовательных Организаций," Вопросы образования // Educational Studies Moscow, National Research University Higher School of Economics, issue 1, pages 97-125.
    2. Mehdi Rhaiem & Nabil Amara, 2020. "Determinants of research efficiency in Canadian business schools: evidence from scholar-level data," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 53-99, October.
    3. Matthias Menter & Erik E. Lehmann & Torben Klarl, 2018. "In search of excellence: a case study of the first excellence initiative of Germany," Journal of Business Economics, Springer, vol. 88(9), pages 1105-1132, December.
    4. Yu, Nannan & Dong, Yueyan & de Jong, Martin, 2022. "A helping hand from the government? How public research funding affects academic output in less-prestigious universities in China," Research Policy, Elsevier, vol. 51(10).
    5. Erik E. Lehmann & Michele Meoli & Stefano Paleari & Sarah A. E. Stockinger, 2018. "Approaching effects of the economic crisis on university efficiency: a comparative study of Germany and Italy," Eurasian Business Review, Springer;Eurasia Business and Economics Society, vol. 8(1), pages 37-54, March.
    6. Nataliya Matveeva & Anuška Ferligoj, 2020. "Scientific collaboration in Russian universities before and after the excellence initiative Project 5-100," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(3), pages 2383-2407, September.
    7. Matveeva, Nataliya & Sterligov, Ivan & Yudkevich, Maria, 2021. "The effect of Russian University Excellence Initiative on publications and collaboration patterns," Journal of Informetrics, Elsevier, vol. 15(1).
    8. Manuel Salas-Velasco, 2020. "Measuring and explaining the production efficiency of Spanish universities using a non-parametric approach and a bootstrapped-truncated regression," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(2), pages 825-846, February.
    9. Nataliya Matveeva & Ivan Sterligov & Maria Yudkevich, 2019. "The Russian University Excellence Initiative: Is It Really Excellence That Is Promoted?," HSE Working papers WP BRP 49/EDU/2019, National Research University Higher School of Economics.
    10. Arie Y Lewin & Silvia Massini & Carine Peeters, 2020. "Absorptive capacity, socially enabling mechanisms, and the role of learning from trial and error experiments: A tribute to Dan Levinthal’s contribution to international business research," Journal of International Business Studies, Palgrave Macmillan;Academy of International Business, vol. 51(9), pages 1568-1579, December.
    11. Elena Madeo, 2021. "The Role of Crowdfunding for New Funding Challenges in Public Universities: An Italian Case Study," Journal of Education for Sustainable Development, , vol. 15(2), pages 186-205, September.
    12. Ieda Margarete Oro & Carlos Eduardo Facin Lavarda, 2017. "Interaction Between Strategy and Organizational Performance: The Influence of Family Management," Brazilian Business Review, Fucape Business School, vol. 14(5), pages 493-509, September.
    13. Schubert, Torben, 2009. "Empirical observations on New Public Management to increase efficiency in public research--Boon or bane?," Research Policy, Elsevier, vol. 38(8), pages 1225-1234, October.
    14. Pontus Mattsson & Jonas Månsson & Christian Andersson & Fredrik Bonander, 2018. "A bootstrapped Malmquist index applied to Swedish district courts," European Journal of Law and Economics, Springer, vol. 46(1), pages 109-139, August.
    15. Andrews, Matt & Pritchett, Lant & Woolcock, Michael, 2013. "Escaping Capability Traps Through Problem Driven Iterative Adaptation (PDIA)," World Development, Elsevier, vol. 51(C), pages 234-244.
    16. Audretsch, David B. & Lehmann, Erik E. & Warning, Susanne, 2005. "University spillovers and new firm location," Research Policy, Elsevier, vol. 34(7), pages 1113-1122, September.
    17. Chen, Guanghua & Yang, Guoliang & He, Feng & Chen, Kaihua, 2019. "Exploring the effect of political borders on university-industry collaborative research performance: Evidence from China’s Guangdong province," Technovation, Elsevier, vol. 82, pages 58-69.
    18. Vittadini, Giorgio & Sturaro, Caterina & Folloni, Giuseppe, 2022. "Non-Cognitive Skills and Cognitive Skills to measure school efficiency," Socio-Economic Planning Sciences, Elsevier, vol. 81(C).
    19. repec:unu:wpaper:wp2012-64 is not listed on IDEAS
    20. Qiu, Shumin & Liu, Xielin & Gao, Taishan, 2017. "Do emerging countries prefer local knowledge or distant knowledge? Spillover effect of university collaborations on local firms," Research Policy, Elsevier, vol. 46(7), pages 1299-1311.
    21. Al-Atwi, Amer Ali & Amankwah-Amoah, Joseph & Khan, Zaheer, 2021. "Micro-foundations of organizational design and sustainability: The mediating role of learning ambidexterity," International Business Review, Elsevier, vol. 30(1).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nos:voprob:2020:i:1:p:97-125. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Marta Morozova (email available below). General contact details of provider: http://vo.hse.ru/en/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.