IDEAS home Printed from
   My bibliography  Save this article

Measuring Efficiency: A Comparison of Multilevel Modelling and Data Envelopment Analysis in the Context of Higher Education


  • Jill Johnes


Data envelopment analysis (DEA) and multilevel modelling (MLM) are applied to a data set of 54,564 graduates from UK universities in 1993 to assess whether the choice of technique affects the measurement of universities' performance. A methodology developed by Thanassoulis and Portela (2002; Education Economics, 10(2), pp. 183-207) allows each individual's DEA efficiency score to be decomposed into two components: one attributable to the university at which the student studied and the other attributable to the individual student. From the former component, a measure of each institution's teaching efficiency is derived and compared to the university effects from various multilevel models. The comparisons are made within four broad subjects: pure science, applied science, social science and arts. The results show that the rankings of universities derived from the DEA efficiencies which measure the universities' own performance (i.e., having excluded the efforts of the individuals) are not strongly correlated with the university rankings derived from the university effects of the multilevel models. The data were also used to perform a university-level DEA. The university efficiency scores derived from these DEAs are largely unrelated to the scores from the individual-level DEAs, confirming a result from a smaller data set (Johnes, 2006a; European Journal of Operational Research, forthcoming). However, the university-level DEAs provide efficiency scores which are generally strongly related to the university effects of the multilevel models. Copyright Blackwell Publishers Ltd and the Board of Trustees of the Bulletin of Economic Research, 2006.

Suggested Citation

  • Jill Johnes, 2006. "Measuring Efficiency: A Comparison of Multilevel Modelling and Data Envelopment Analysis in the Context of Higher Education," Bulletin of Economic Research, Wiley Blackwell, vol. 58(2), pages 75-104, April.
  • Handle: RePEc:bla:buecrs:v:58:y:2006:i:2:p:75-104

    Download full text from publisher

    File URL:
    File Function: link to full text
    Download Restriction: Access to full text is restricted to subscribers.

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    1. Hodrick, Robert J & Kocherlakota, Narayana R & Lucas, Deborah, 1991. "The Variability of Velocity in Cash-in-Advance Models," Journal of Political Economy, University of Chicago Press, vol. 99(2), pages 358-384, April.
    2. Petrovic, Pavle & Mladenovic, Zorica, 2000. "Money Demand and Exchange Rate Determination under Hyperinflation: Conceptual Issues and Evidence from Yugoslavia," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 32(4), pages 785-806, November.
    3. Taylor, John B., 1999. "The robustness and efficiency of monetary policy rules as guidelines for interest rate setting by the European central bank," Journal of Monetary Economics, Elsevier, vol. 43(3), pages 655-679, June.
    4. Ireland, Peter N, 1995. "Endogenous Financial Innovation and the Demand for Money," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 27(1), pages 107-123, February.
    5. Hylleberg, S. & Engle, R. F. & Granger, C. W. J. & Yoo, B. S., 1990. "Seasonal integration and cointegration," Journal of Econometrics, Elsevier, vol. 44(1-2), pages 215-238.
    6. Robert W. Dimand, 1999. "Irving Fisher and the Fisher Relation: Setting the Record Straight," Canadian Journal of Economics, Canadian Economics Association, vol. 32(3), pages 744-750, May.
    7. Culver, Sarah E & Papell, David H, 1997. "Is There a Unit Root in the Inflation Rate? Evidence from Sequential Break and Panel Data Models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 12(4), pages 435-444, July-Aug..
    8. Sargent, Thomas J, 1973. "Interest Rates and Prices in the Long Run: A Study of the Gibson Paradox," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 5(1), pages 385-449, Part II F.
    9. Andreas Schabert, 2003. "On the Relevance of Open Market Operations," Working Paper Series in Economics 4, University of Cologne, Department of Economics.
    10. William J. Crowder, 1997. "The Long-Run Fisher Relation in Canada," Canadian Journal of Economics, Canadian Economics Association, vol. 30(4), pages 1124-1142, November.
    11. Max Gillman & Michal Kejak, 2005. "Inflation and Balanced-Path Growth with Alternative Payment Mechanisms," Economic Journal, Royal Economic Society, vol. 115(500), pages 247-270, January.
    12. Carl E. Walsh, 2003. "Monetary Theory and Policy, 2nd Edition," MIT Press Books, The MIT Press, edition 2, volume 1, number 0262232316, January.
    13. Claus Brand & Nuno Cassola, 2004. "A money demand system for euro area M3," Applied Economics, Taylor & Francis Journals, vol. 36(8), pages 817-838.
    14. Crowder, William J & Hoffman, Dennis L, 1996. "The Long-Run Relationship between Nominal Interest Rates and Inflation: The Fisher Equation Revisited," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 28(1), pages 102-118, February.
    15. Canova, Fabio & Hansen, Bruce E, 1995. "Are Seasonal Patterns Constant over Time? A Test for Seasonal Stability," Journal of Business & Economic Statistics, American Statistical Association, vol. 13(3), pages 237-252, July.
    16. Nelson C. Mark & Donggyu Sul, 2003. "Cointegration Vector Estimation by Panel DOLS and Long-run Money Demand," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 65(5), pages 655-680, December.
    17. Joseph Beaulieu, J. & Miron, Jeffrey A., 1993. "Seasonal unit roots in aggregate U.S. data," Journal of Econometrics, Elsevier, vol. 55(1-2), pages 305-328.
    18. Perron, Pierre, 1989. "The Great Crash, the Oil Price Shock, and the Unit Root Hypothesis," Econometrica, Econometric Society, vol. 57(6), pages 1361-1401, November.
    19. Peter N. Ireland, 2001. "The Real Balance Effect," Boston College Working Papers in Economics 491, Boston College Department of Economics.
    20. Gillman M. & Siklos & P.L.Silver & J.L., 1996. "Money Velocity with Costly Credit," Department of Economics - Working Papers Series 515, The University of Melbourne.
    21. Max Gillman & Michal Kejak, 2004. "The Demand for Bank Reserves and Other Monetary Aggregates," Economic Inquiry, Western Economic Association International, vol. 42(3), pages 518-533, July.
    22. Claus Brand & Dieter Gerdesmeier & Barbara Roffia, 2002. "Estimating the trend of M3 income velocity underlying the reference value for monetary growth," Occasional Paper Series 03, European Central Bank.
    23. Gillman, Max & Otto, Glenn, 2003. "Money demand in a banking time economy," HWWA Discussion Papers 254, Hamburg Institute of International Economics (HWWA).
    24. Hoffman, Dennis L. & Rasche, Robert H. & Tieslau, Margie A., 1995. "The stability of long-run money demand in five industrial countries," Journal of Monetary Economics, Elsevier, vol. 35(2), pages 317-339, April.
    25. Michal Kejak & Szilard Benk & Max Gillman, 2004. "Credit Shocks in a Monetary Business Cycle," 2004 Meeting Papers 133, Society for Economic Dynamics.
    26. Billmeier, Andreas & Bonato, Leo, 2004. "Exchange rate pass-through and monetary policy in Croatia," Journal of Comparative Economics, Elsevier, vol. 32(3), pages 426-444, September.
    27. Davidson, James E H, et al, 1978. "Econometric Modelling of the Aggregate Time-Series Relationship between Consumers' Expenditure and Income in the United Kingdom," Economic Journal, Royal Economic Society, vol. 88(352), pages 661-692, December.
    28. Johansen, Soren, 1995. "Likelihood-Based Inference in Cointegrated Vector Autoregressive Models," OUP Catalogue, Oxford University Press, number 9780198774501, June.
    29. Ireland, Peter N, 1996. "The Role of Countercyclical Monetary Policy," Journal of Political Economy, University of Chicago Press, vol. 104(4), pages 704-723, August.
    30. Evan Kraft, 2003. "Monetary Policy under Dollarisation: The Case of Croatia," Comparative Economic Studies, Palgrave Macmillan;Association for Comparative Economic Studies, vol. 45(3), pages 256-277, September.
    31. John B. Taylor, 2001. "The Role of the Exchange Rate in Monetary-Policy Rules," American Economic Review, American Economic Association, vol. 91(2), pages 263-267, May.
    32. Kydland, Finn E & Prescott, Edward C, 1977. "Rules Rather Than Discretion: The Inconsistency of Optimal Plans," Journal of Political Economy, University of Chicago Press, vol. 85(3), pages 473-491, June.
    33. Lucas, Robert E, Jr, 1980. "Equilibrium in a Pure Currency Economy," Economic Inquiry, Western Economic Association International, vol. 18(2), pages 203-220, April.
    Full references (including those not matched with items on IDEAS)


    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.

    Cited by:

    1. C F Chen & K T Soo, 2009. "Some university students are more equal than others: Evidence from England," Working Papers 600480, Lancaster University Management School, Economics Department.
    2. Kristof de Witte & Laura López-Torres, 2015. "Efficiency in Education. A Review of Literature and a Way Forward," Working Papers 1501, Departament Empresa, Universitat Autònoma de Barcelona, revised Apr 2015.
    3. Jean-Charles Billaut & Denis Bouyssou & Philippe Vincke, 2009. "Should you believe in the Shanghai ranking? An MCDM view," Working Papers hal-00877050, HAL.
    4. Yaisawarng, Suthathip & Ng, Ying Chu, 2014. "The impact of higher education reform on research performance of Chinese universities," China Economic Review, Elsevier, vol. 31(C), pages 94-105.
    5. Billaut, Jean-Charles & Bouyssou, Denis & Vincke, Philippe, 2010. "Faut-il croire le classement de Shangaï ?," Revue de la Régulation - Capitalisme, institutions, pouvoirs, Association Recherche et Régulation, vol. 8.
    6. repec:pal:jorsoc:v:61:y:2010:i:8:d:10.1057_jors.2009.50 is not listed on IDEAS
    7. Aleksandra Parteka & Joanna Wolszczak-Derlacz, 2013. "Dynamics of productivity in higher education: cross-european evidence based on bootstrapped Malmquist indices," Journal of Productivity Analysis, Springer, vol. 40(1), pages 67-82, August.
    8. Deutsch, Joseph & Dumas, Audrey & Silber, Jacques, 2013. "Estimating an educational production function for five countries of Latin America on the basis of the PISA data," Economics of Education Review, Elsevier, vol. 36(C), pages 245-262.
    9. Jean-Charles Billaut & Denis Bouyssou & Philippe Vincke, 2010. "Should you believe in the Shanghai ranking?," Post-Print hal-00388319, HAL.
    10. Angel García Correas & Manuel Larran Jorge, 2010. "Análisis de diferentes medidas de la eficiencia investigadora y factores explicativos en las universidades públicas españolas," Investigaciones de Economía de la Educación volume 5,in: María Jesús Mancebón-Torrubia & Domingo P. Ximénez-de-Embún & José María Gómez-Sancho & Gregorio Gim (ed.), Investigaciones de Economía de la Educación 5, edition 1, volume 5, chapter 36, pages 703-732 Asociación de Economía de la Educación.
    11. Kong, Wei-Hsin & Fu, Tsu-Tan, 2012. "Assessing the performance of business colleges in Taiwan using data envelopment analysis and student based value-added performance indicators," Omega, Elsevier, vol. 40(5), pages 541-549.
    12. Tzeremes, Nickolaos & Halkos, George, 2010. "A DEA approach for measuring university departments’ efficiency," MPRA Paper 24029, University Library of Munich, Germany.
    13. George E. Halkos & Nickolaos G. Tzeremes & Stavros A. Kourtzidis, 2012. "Measuring Public Owned University Departments' Efficiency: A Bootstrapped DEA Approach," Journal of Economics and Econometrics, Economics and Econometrics Society, vol. 55(2), pages 1-24.
    14. Halkos, George & Tzeremes, Nickolaos, 2011. "Adjusting for cultural effects on countries’ education policy efficiency:an application of conditional full frontiers measures," MPRA Paper 30098, University Library of Munich, Germany.
    15. Tommaso Agasisti, 2014. "How does schools’ efficiency look like across Europe? An empirical analysis of Germany, Spain, France, Italy and UK using OECD PISA2012 data," Working papers 9, Società Italiana di Economia Pubblica.
    16. repec:spr:scient:v:84:y:2010:i:1:d:10.1007_s11192-009-0115-x is not listed on IDEAS
    17. repec:spr:scient:v:95:y:2013:i:2:d:10.1007_s11192-012-0844-0 is not listed on IDEAS

    More about this item


    Access and download statistics


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:buecrs:v:58:y:2006:i:2:p:75-104. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Wiley-Blackwell Digital Licensing) or (Christopher F. Baum). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.