IDEAS home Printed from https://ideas.repec.org/a/bla/buecrs/v58y2006i2p75-104.html
   My bibliography  Save this article

Measuring Efficiency: A Comparison of Multilevel Modelling and Data Envelopment Analysis in the Context of Higher Education

Author

Listed:
  • Jill Johnes

Abstract

Data envelopment analysis (DEA) and multilevel modelling (MLM) are applied to a data set of 54,564 graduates from UK universities in 1993 to assess whether the choice of technique affects the measurement of universities’ performance. A methodology developed by Thanassoulis and Portela (2002; Education Economics, 10(2), pp. 183–207) allows each individual's DEA efficiency score to be decomposed into two components: one attributable to the university at which the student studied and the other attributable to the individual student. From the former component, a measure of each institution's teaching efficiency is derived and compared to the university effects from various multilevel models. The comparisons are made within four broad subjects: pure science, applied science, social science and arts. The results show that the rankings of universities derived from the DEA efficiencies which measure the universities’ own performance (i.e., having excluded the efforts of the individuals) are not strongly correlated with the university rankings derived from the university effects of the multilevel models. The data were also used to perform a university‐level DEA. The university efficiency scores derived from these DEAs are largely unrelated to the scores from the individual‐level DEAs, confirming a result from a smaller data set (Johnes, 2006a; European Journal of Operational Research, forthcoming). However, the university‐level DEAs provide efficiency scores which are generally strongly related to the university effects of the multilevel models.

Suggested Citation

  • Jill Johnes, 2006. "Measuring Efficiency: A Comparison of Multilevel Modelling and Data Envelopment Analysis in the Context of Higher Education," Bulletin of Economic Research, Wiley Blackwell, vol. 58(2), pages 75-104, April.
  • Handle: RePEc:bla:buecrs:v:58:y:2006:i:2:p:75-104
    DOI: 10.1111/j.0307-3378.2006.00238.x
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/j.0307-3378.2006.00238.x
    Download Restriction: no

    File URL: https://libkey.io/10.1111/j.0307-3378.2006.00238.x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Forsund, Finn R. & Lovell, C. A. Knox & Schmidt, Peter, 1980. "A survey of frontier production functions and of their relationship to efficiency measurement," Journal of Econometrics, Elsevier, vol. 13(1), pages 5-25, May.
    2. Bratti, Massimiliano, 2002. "Does the choice of university matter?: a study of the differences across UK universities in life sciences students' degree performance," Economics of Education Review, Elsevier, vol. 21(5), pages 431-443, October.
    3. Philip Andrew Stevens, 2005. "A Stochastic Frontier Analysis of English and Welsh Universities," Education Economics, Taylor & Francis Journals, vol. 13(4), pages 355-374.
    4. Johnes, Jill, 1996. "Performance assessment in higher education in Britain," European Journal of Operational Research, Elsevier, vol. 89(1), pages 18-33, February.
    5. Massimiliano Bratti & Abigail McKnight & Robin Naylor & Jeremy Smith, 2004. "Higher education outcomes, graduate employment and university performance indicators," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 167(3), pages 475-496, August.
    6. Jeremy P. Smith & Robin A. Naylor, 2001. "Dropping out of university: A statistical analysis of the probability of withdrawal for UK university students," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 164(2), pages 389-405.
    7. Barrow, Michael M., 1991. "Measuring local education authority performance: A frontier approach," Economics of Education Review, Elsevier, vol. 10(1), pages 19-27, March.
    8. Antreas Athanassopoulos & Estelle Shale, 1997. "Assessing the Comparative Efficiency of Higher Education Institutions in the UK by the Means of Data Envelopment Analysis," Education Economics, Taylor & Francis Journals, vol. 5(2), pages 117-134.
    9. Gary Madden & Scott Savage & Steven Kemp, 1997. "Measuring Public Sector Efficiency: A Study of Economics Departments at Australian Universities," Education Economics, Taylor & Francis Journals, vol. 5(2), pages 153-168.
    10. Emmanuel Thanassoulis & Maria Da Conceicao & A. Silva Portela, 2002. "School Outcomes: Sharing the Responsibility Between Pupil and School1," Education Economics, Taylor & Francis Journals, vol. 10(2), pages 183-207.
    11. Deller, Steven C. & Rudnicki, Edward, 1993. "Production efficiency in elementary education: The case of Maine public schools," Economics of Education Review, Elsevier, vol. 12(1), pages 45-57, March.
    12. Izadi, Hooshang & Johnes, Geraint & Oskrochi, Reza & Crouchley, Robert, 2002. "Stochastic frontier estimation of a CES cost function: the case of higher education in Britain," Economics of Education Review, Elsevier, vol. 21(1), pages 63-71, February.
    13. Geraint Johnes, 1999. "The Management of Universities; Scottish Economic Society/Royal Bank of Scotland Annual Lecture, 1999," Scottish Journal of Political Economy, Scottish Economic Society, vol. 46(5), pages 505-522, November.
    14. Johnes, Jill, 2006. "Data envelopment analysis and its application to the measurement of efficiency in higher education," Economics of Education Review, Elsevier, vol. 25(3), pages 273-288, June.
    15. Korhonen, Pekka & Tainio, Risto & Wallenius, Jyrki, 2001. "Value efficiency analysis of academic research," European Journal of Operational Research, Elsevier, vol. 130(1), pages 121-132, April.
    16. Avkiran, Necmi K., 2001. "Investigating technical and scale efficiencies of Australian Universities through data envelopment analysis," Socio-Economic Planning Sciences, Elsevier, vol. 35(1), pages 57-80, March.
    17. Cuesta, Rafael A. & Orea, Luis, 2002. "Mergers and technical efficiency in Spanish savings banks: A stochastic distance function approach," Journal of Banking & Finance, Elsevier, vol. 26(12), pages 2231-2247.
    18. El-Mahgary, Sami & Lahdelma, Risto, 1995. "Data envelopment analysis: Visualizing the results," European Journal of Operational Research, Elsevier, vol. 83(3), pages 700-710, June.
    19. Johnes, Jill, 2006. "Measuring teaching efficiency in higher education: An application of data envelopment analysis to economics graduates from UK Universities 1993," European Journal of Operational Research, Elsevier, vol. 174(1), pages 443-456, October.
    20. Harvey Goldstein & David J. Spiegelhalter, 1996. "League Tables and Their Limitations: Statistical Issues in Comparisons of Institutional Performance," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 159(3), pages 385-409, May.
    21. Jeremy Smith & Robin Naylor, 2001. "Determinants of Degree Performance in UK Universities: A Statistical Analysis of the 1993 Student Cohort," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 63(1), pages 29-60, February.
    22. Fried, Harold O. & Lovell, C. A. Knox & Schmidt, Shelton S. (ed.), 1993. "The Measurement of Productive Efficiency: Techniques and Applications," OUP Catalogue, Oxford University Press, number 9780195072181.
    23. Smith, Jeremy & McKnight, Abigail & Naylor, Robin, 2000. "Graduate Employability: Policy and Performance in Higher Education in the UK," Economic Journal, Royal Economic Society, vol. 110(464), pages 382-411, June.
    24. Abbott, M. & Doucouliagos, C., 2003. "The efficiency of Australian universities: a data envelopment analysis," Economics of Education Review, Elsevier, vol. 22(1), pages 89-97, February.
    25. Aigner, Dennis & Lovell, C. A. Knox & Schmidt, Peter, 1977. "Formulation and estimation of stochastic frontier production function models," Journal of Econometrics, Elsevier, vol. 6(1), pages 21-37, July.
    26. Silva Portela, Maria Conceicao A. & Thanassoulis, Emmanuel, 2001. "Decomposing school and school-type efficiency," European Journal of Operational Research, Elsevier, vol. 132(2), pages 357-373, July.
    27. Harvey Goldstein & Sally Thomas, 1996. "Using Examination Results as Indicators of School and College Performance," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 159(1), pages 149-163, January.
    28. Johnes, Geraint, 1998. "The Costs of Multi-product Organizations and the Heuristic Evaluation of Industrial Structure," Socio-Economic Planning Sciences, Elsevier, vol. 32(3), pages 199-209, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jean-Charles Billaut & Denis Bouyssou & Philippe Vincke, 2010. "Should you believe in the Shanghai ranking?," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(1), pages 237-263, July.
    2. Billaut, Jean-Charles & Bouyssou, Denis & Vincke, Philippe, 2010. "Faut-il croire le classement de Shangaï ?," Revue de la Régulation - Capitalisme, institutions, pouvoirs, Association Recherche et Régulation, vol. 8.
    3. K De Witte & E Thanassoulis & G Simpson & G Battisti & A Charlesworth-May, 2010. "Assessing pupil and school performance by non-parametric and parametric techniques," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 61(8), pages 1224-1237, August.
    4. C F Chen & K T Soo, 2009. "Some university students are more equal than others: Evidence from England," Working Papers 600480, Lancaster University Management School, Economics Department.
    5. Guironnet, Jean-Pascal & Peypoch, Nicolas, 2018. "The geographical efficiency of education and research: The ranking of U.S. universities," Socio-Economic Planning Sciences, Elsevier, vol. 62(C), pages 44-55.
    6. Jie Wu & Ganggang Zhang & Qingyuan Zhu & Zhixiang Zhou, 2020. "An efficiency analysis of higher education institutions in China from a regional perspective considering the external environmental impact," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(1), pages 57-70, January.
    7. Halkos, George & Tzeremes, Nickolaos, 2011. "Adjusting for cultural effects on countries’ education policy efficiency:an application of conditional full frontiers measures," MPRA Paper 30098, University Library of Munich, Germany.
    8. M. L. Bougnol & J. H. Dulá, 2013. "A mathematical model to optimize decisions to impact multi-attribute rankings," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(2), pages 785-796, May.
    9. Angel García Correas & Manuel Larran Jorge, 2010. "Análisis de diferentes medidas de la eficiencia investigadora y factores explicativos en las universidades públicas españolas," Investigaciones de Economía de la Educación volume 5, in: María Jesús Mancebón-Torrubia & Domingo P. Ximénez-de-Embún & José María Gómez-Sancho & Gregorio Gim (ed.), Investigaciones de Economía de la Educación 5, edition 1, volume 5, chapter 36, pages 703-732, Asociación de Economía de la Educación.
    10. Kristof De Witte & Laura López-Torres, 2017. "Efficiency in education: a review of literature and a way forward," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 68(4), pages 339-363, April.
    11. Abro, Z.A. & Debela, B.L., 2018. "Technical change through crop improvement: are there synergies or tradeoffs in land productivity and efficiency?," 2018 Conference, July 28-August 2, 2018, Vancouver, British Columbia 277086, International Association of Agricultural Economists.
    12. Jean-Charles Billaut & Denis Bouyssou & Philippe Vincke, 2009. "Should you believe in the Shanghai ranking? An MCDM view," Working Papers hal-00877050, HAL.
    13. John V.C. Nye & Cheryl Litman & Maksym Bryukhanov & Sergiy Polyachenko, 2020. "The Universal Link between Higher Education and Pro-Market Values," HSE Working papers WP BRP 225/EC/2020, National Research University Higher School of Economics.
    14. Tommaso Agasisti & Aleksei Egorov & Margarita Maximova, 2020. "Do Merger Policies Increase University Efficiency? Evidence From A Fuzzy Regression Discontinuity Design," HSE Working papers WP BRP 226/EC/2020, National Research University Higher School of Economics.
    15. Sami El-Mahgary & Petri Rönnholm & Hannu Hyyppä & Henrik Haggrén & Jenni Koponen, 2014. "Evaluating the performance of university course units using data envelopment analysis," Cogent Economics & Finance, Taylor & Francis Journals, vol. 2(1), pages 1-20, December.
    16. Tzeremes, Nickolaos & Halkos, George, 2010. "A DEA approach for measuring university departments’ efficiency," MPRA Paper 24029, University Library of Munich, Germany.
    17. Aleksandra Parteka & Joanna Wolszczak-Derlacz, 2013. "Dynamics of productivity in higher education: cross-european evidence based on bootstrapped Malmquist indices," Journal of Productivity Analysis, Springer, vol. 40(1), pages 67-82, August.
    18. Kong, Wei-Hsin & Fu, Tsu-Tan, 2012. "Assessing the performance of business colleges in Taiwan using data envelopment analysis and student based value-added performance indicators," Omega, Elsevier, vol. 40(5), pages 541-549.
    19. Yaisawarng, Suthathip & Ng, Ying Chu, 2014. "The impact of higher education reform on research performance of Chinese universities," China Economic Review, Elsevier, vol. 31(C), pages 94-105.
    20. George E. Halkos & Nickolaos G. Tzeremes & Stavros A. Kourtzidis, 2012. "Measuring Public Owned University Departments' Efficiency: A Bootstrapped DEA Approach," Journal of Economics and Econometrics, Economics and Econometrics Society, vol. 55(2), pages 1-24.
    21. Cristian Barra & Roberto Zotti, 2017. "Investigating the Human Capital Development–growth Nexus," International Regional Science Review, , vol. 40(6), pages 638-678, November.
    22. Tommaso Agasisti, 2014. "How does schools’ efficiency look like across Europe? An empirical analysis of Germany, Spain, France, Italy and UK using OECD PISA2012 data," Working papers 9, Società Italiana di Economia Pubblica.
    23. Deutsch, Joseph & Dumas, Audrey & Silber, Jacques, 2013. "Estimating an educational production function for five countries of Latin America on the basis of the PISA data," Economics of Education Review, Elsevier, vol. 36(C), pages 245-262.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kristof De Witte & Laura López-Torres, 2017. "Efficiency in education: a review of literature and a way forward," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 68(4), pages 339-363, April.
    2. Johnes, Jill, 2006. "Data envelopment analysis and its application to the measurement of efficiency in higher education," Economics of Education Review, Elsevier, vol. 25(3), pages 273-288, June.
    3. Mehdi Rhaiem, 2017. "Measurement and determinants of academic research efficiency: a systematic review of the evidence," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(2), pages 581-615, February.
    4. Guironnet, Jean-Pascal & Peypoch, Nicolas, 2018. "The geographical efficiency of education and research: The ranking of U.S. universities," Socio-Economic Planning Sciences, Elsevier, vol. 62(C), pages 44-55.
    5. Agasisti, Tommaso & Barra, Cristian & Zotti, Roberto, 2016. "Evaluating the efficiency of Italian public universities (2008–2011) in presence of (unobserved) heterogeneity," Socio-Economic Planning Sciences, Elsevier, vol. 55(C), pages 47-58.
    6. Laureti, Tiziana & Secondi, Luca & Biggeri, Luigi, 2014. "Measuring the efficiency of teaching activities in Italian universities: An information theoretic approach," Economics of Education Review, Elsevier, vol. 42(C), pages 147-164.
    7. Johnes, Jill & Yu, Li, 2008. "Measuring the research performance of Chinese higher education institutions using data envelopment analysis," China Economic Review, Elsevier, vol. 19(4), pages 679-696, December.
    8. Barra, Cristian & Lagravinese, Raffaele & Zotti, Roberto, 2015. "Explaining (in)efficiency in higher education: a comparison of parametric and non-parametric analyses to rank universities," MPRA Paper 67119, University Library of Munich, Germany.
    9. Johnes, Jill, 2006. "Measuring teaching efficiency in higher education: An application of data envelopment analysis to economics graduates from UK Universities 1993," European Journal of Operational Research, Elsevier, vol. 174(1), pages 443-456, October.
    10. Barra, Cristian & Lagravinese, Raffaele & Zotti, Roberto, 2018. "Does econometric methodology matter to rank universities? An analysis of Italian higher education system," Socio-Economic Planning Sciences, Elsevier, vol. 62(C), pages 104-120.
    11. Johnes, Jill, 2015. "Operational Research in education," European Journal of Operational Research, Elsevier, vol. 243(3), pages 683-696.
    12. Tzeremes, Nickolaos & Halkos, George, 2010. "A DEA approach for measuring university departments’ efficiency," MPRA Paper 24029, University Library of Munich, Germany.
    13. Martín Rivero, Raquel, 2007. "La eficiencia productiva en el ámbito universitario: aspectos claves para su evaluación/The Productive Efficiency in the University: Key Aspects for its Evaluation," Estudios de Economia Aplicada, Estudios de Economia Aplicada, vol. 25, pages 793-812, Diciembre.
    14. Mad Ithnin Salleh & Nurul Fadly Habidin & Abdul Halim Masnan & Nordin Mamat, 2017. "Estimating Technical Efficiency and Bootstrapping Malmquist Indices: Analysis of Malaysian Preschool Sector," International Journal of Academic Research in Business and Social Sciences, Human Resource Management Academic Research Society, International Journal of Academic Research in Business and Social Sciences, vol. 7(3), pages 440-457, March.
    15. Yaisawarng, Suthathip & Ng, Ying Chu, 2014. "The impact of higher education reform on research performance of Chinese universities," China Economic Review, Elsevier, vol. 31(C), pages 94-105.
    16. Lars Herberholz & Berthold U. Wigger, 2020. "Efficiency of European Universities: A Comparison of Peers," CESifo Working Paper Series 8044, CESifo.
    17. Fatih KARASAÇ, 2020. "Efficiency Analysis of Turkish Higher Education Institutions: Stochastic Frontier Model," Sosyoekonomi Journal, Sosyoekonomi Society, issue 28(44).
    18. Calogero Guccio & Marco Ferdinando Martorana & Isidoro Mazza, 2016. "Efficiency assessment and convergence in teaching and research in Italian public universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(3), pages 1063-1094, June.
    19. Celik, Orhan & Ecer, Alaattin, 2009. "Efficiency in accounting education: evidence from Turkish Universities," CRITICAL PERSPECTIVES ON ACCOUNTING, Elsevier, vol. 20(5), pages 614-634.
    20. Deutsch, Joseph & Dumas, Audrey & Silber, Jacques, 2013. "Estimating an educational production function for five countries of Latin America on the basis of the PISA data," Economics of Education Review, Elsevier, vol. 36(C), pages 245-262.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:buecrs:v:58:y:2006:i:2:p:75-104. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0307-3378 .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0307-3378 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.