IDEAS home Printed from https://ideas.repec.org/a/eee/soceps/v62y2018icp104-120.html
   My bibliography  Save this article

Does econometric methodology matter to rank universities? An analysis of Italian higher education system

Author

Listed:
  • Barra, Cristian
  • Lagravinese, Raffaele
  • Zotti, Roberto

Abstract

In recent years more and more numerous are the rankings published in newspapers or technical reports available, covering many aspects of higher education, but in many cases with very conflicting results between them, due to the fact that universities' performances depend on the set of variables considered and on the methods of analysis employed. This study measures the efficiency of Italian higher education using both parametric and non-parametric techniques and uses the results to provide guidance to university managers and policymakers regarding the most appropriate method for their needs. The findings reveal that, on average and among the macro-areas of the country, the level of efficiency does not change significantly among estimation approaches, which produce different rankings, instead. This may have important implications as rankings have a strong impact on academic decision-making and behaviour, on the structure of the institutions and also on students and graduates recruiters.

Suggested Citation

  • Barra, Cristian & Lagravinese, Raffaele & Zotti, Roberto, 2018. "Does econometric methodology matter to rank universities? An analysis of Italian higher education system," Socio-Economic Planning Sciences, Elsevier, vol. 62(C), pages 104-120.
  • Handle: RePEc:eee:soceps:v:62:y:2018:i:c:p:104-120
    DOI: 10.1016/j.seps.2017.09.002
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0038012117300381
    Download Restriction: Full text for ScienceDirect subscribers only

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Tyagi, Preeti & Yadav, Shiv Prasad & Singh, S.P., 2009. "Relative performance of academic departments using DEA with sensitivity analysis," Evaluation and Program Planning, Elsevier, vol. 32(2), pages 168-177, May.
    2. Johnes, Jill & Yu, Li, 2008. "Measuring the research performance of Chinese higher education institutions using data envelopment analysis," China Economic Review, Elsevier, vol. 19(4), pages 679-696, December.
    3. Johnes, Geraint & Johnes, Jill, 1993. "Measuring the Research Performance of UK Economics Departments: An Application of Data Envelopment Analysis," Oxford Economic Papers, Oxford University Press, vol. 45(2), pages 332-347, April.
    4. Diego A. Restrepo-Tobón & Subal C. Kumbhakar, 2013. "Profit efficiency of U.S. commercial banks: a decomposition," Documentos de Trabajo CIEF 010939, Universidad EAFIT.
    5. Gong, Byeong-Ho & Sickles, Robin C., 1992. "Finite sample evidence on the performance of stochastic frontiers and data envelopment analysis using panel data," Journal of Econometrics, Elsevier, vol. 51(1-2), pages 259-284.
    6. J. M. Bates, 1997. "Measuring predetermined socioeconomic 'inputs' when assessing the efficiency of educational outputs," Applied Economics, Taylor & Francis Journals, vol. 29(1), pages 85-93.
    7. O'Donnell, Christopher J. & Coelli, Timothy J., 2005. "A Bayesian approach to imposing curvature on distance functions," Journal of Econometrics, Elsevier, vol. 126(2), pages 493-523, June.
    8. Maria Olivares & Heike Wetzel, 2014. "Editor's Choice Competing in the Higher Education Market: Empirical Evidence for Economies of Scale and Scope in German Higher Education Institutions," CESifo Economic Studies, CESifo, vol. 60(4), pages 653-680.
    9. repec:spr:scient:v:63:y:2005:i:1:d:10.1007_s11192-005-0205-3 is not listed on IDEAS
    10. Agasisti, Tommaso & Barra, Cristian & Zotti, Roberto, 2016. "Evaluating the efficiency of Italian public universities (2008–2011) in presence of (unobserved) heterogeneity," Socio-Economic Planning Sciences, Elsevier, vol. 55(C), pages 47-58.
    11. Johnes, Jill, 1996. "Performance assessment in higher education in Britain," European Journal of Operational Research, Elsevier, vol. 89(1), pages 18-33, February.
    12. Jill Johnes, 2004. "Efficiency Measurement," Chapters,in: International Handbook on the Economics of Education, chapter 16 Edward Elgar Publishing.
    13. Wiji Arulampalam & Robin A. Naylor & Jeremy P. Smith, 2004. "A hazard model of the probability of medical school drop-out in the UK," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 167(1), pages 157-178.
    14. Cooper, W. W. & Tone, K., 1997. "Measures of inefficiency in data envelopment analysis and stochastic frontier estimation," European Journal of Operational Research, Elsevier, vol. 99(1), pages 72-88, May.
    15. Antreas Athanassopoulos & Estelle Shale, 1997. "Assessing the Comparative Efficiency of Higher Education Institutions in the UK by the Means of Data Envelopment Analysis," Education Economics, Taylor & Francis Journals, vol. 5(2), pages 117-134.
    16. Malcolm Abbott & Chris Doucouliagos, 2009. "Competition and efficiency: overseas students and technical efficiency in Australian and New Zealand universities," Education Economics, Taylor & Francis Journals, vol. 17(1), pages 31-57.
    17. Gary Madden & Scott Savage & Steven Kemp, 1997. "Measuring Public Sector Efficiency: A Study of Economics Departments at Australian Universities," Education Economics, Taylor & Francis Journals, vol. 5(2), pages 153-168.
    18. Catherine J. Morrison Paul & Warren E. Johnston & Gerald A. G. Frengley, 2000. "Efficiency in New Zealand Sheep and Beef Farming: The Impacts of Regulatory Reform," The Review of Economics and Statistics, MIT Press, vol. 82(2), pages 325-337, May.
    19. Tommaso Agasisti & Geraint Johnes, 2010. "Heterogeneity and the evaluation of efficiency: the case of Italian universities," Applied Economics, Taylor & Francis Journals, vol. 42(11), pages 1365-1375.
    20. Patrizia Ordine & Claudio Lupi, 2009. "Family Income and Students' Mobility," Giornale degli Economisti, GDE (Giornale degli Economisti e Annali di Economia), Bocconi University, vol. 68(1), pages 1-23, April.
    21. Liang Zhang & Wei Bao & Liang Sun, 2016. "Resources and Research Production in Higher Education: A Longitudinal Analysis of Chinese Universities, 2000–2010," Research in Higher Education, Springer;Association for Institutional Research, vol. 57(7), pages 869-891, November.
    22. T Agasisti & G Johnes, 2009. "Cost Structure, Efficiency and Heterogeneity in US Higher Education: An Empirical Analysis," Working Papers 599308, Lancaster University Management School, Economics Department.
    23. Gerhard Kempkes & Carsten Pohl, 2010. "The efficiency of German universities-some evidence from nonparametric and parametric methods," Applied Economics, Taylor & Francis Journals, vol. 42(16), pages 2063-2079.
    24. Claudia Curi & Cinzia Daraio & Patrick Llerena, 2012. "University technology transfer: how (in)efficient are French universities?," Cambridge Journal of Economics, Oxford University Press, vol. 36(3), pages 629-654.
    25. G. Boero & A. Mcknight & R. Naylor & J. Smith, 2001. "Graduates and graduate labour markets in the UK and Italy," Working Paper CRENoS 200111, Centre for North South Economic Research, University of Cagliari and Sassari, Sardinia.
    26. COELLI, Tim, 2000. "On the econometric estimation of the distance function representation of a production technology," CORE Discussion Papers 2000042, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    27. Jill Johnes, 2014. "Efficiency and Mergers in English Higher Education 1996/97 to 2008/9: Parametric and Non-parametric Estimation of the Multi-input Multi-output Distance Function," Manchester School, University of Manchester, vol. 82(4), pages 465-487, July.
    28. Kristof de Witte & Laura López-Torres, 2015. "Efficiency in Education. A Review of Literature and a Way Forward," Working Papers 1501, Departament Empresa, Universitat Autònoma de Barcelona, revised Apr 2015.
    29. Agasisti, Tommaso & Wolszczak-Derlacz, Joana, 2014. "Exploring universities’ efficiency differentials between countries in a multi-year perspective: an application of bootstrap DEA and Malmquist index to Italy and Poland, 2001-2011," Institute for Research on Labor and Employment, Working Paper Series qt14m8g45v, Institute of Industrial Relations, UC Berkeley.
    30. Maria Alberta Oliveira & Carlos Santos, 2005. "Assessing school efficiency in Portugal using FDH and bootstrapping," Applied Economics, Taylor & Francis Journals, vol. 37(8), pages 957-968.
    31. Hashimoto, Akihiro & Haneda, Shoko, 2008. "Measuring the change in R&D efficiency of the Japanese pharmaceutical industry," Research Policy, Elsevier, vol. 37(10), pages 1829-1836, December.
    32. Bruno S. Frey & Katja Rost, 2010. "Do rankings reflect research quality?," Journal of Applied Economics, Universidad del CEMA, vol. 13, pages 1-38, May.
    33. Maietta, Ornella Wanda, 2015. "Determinants of university–firm R&D collaboration and its impact on innovation: A perspective from a low-tech industry," Research Policy, Elsevier, vol. 44(7), pages 1341-1359.
    34. repec:spr:scient:v:89:y:2011:i:3:d:10.1007_s11192-011-0484-9 is not listed on IDEAS
    35. de Groot, Hans & McMahon, Walter W & Volkwein, J Fredericks, 1991. "The Cost Structure of American Research Universities," The Review of Economics and Statistics, MIT Press, vol. 73(3), pages 424-431, August.
    36. Ellen Hazelkorn, 2007. "The Impact of League Tables and Ranking Systems on Higher Education Decision Making," Higher Education Management and Policy, OECD Publishing, vol. 19(2), pages 1-24.
    37. Worthington, Andrew C. & Lee, Boon L., 2008. "Efficiency, technology and productivity change in Australian universities, 1998-2003," Economics of Education Review, Elsevier, vol. 27(3), pages 285-298, June.
    38. Antonio Alvarez & Christine Amsler & Luis Orea & Peter Schmidt, 2006. "Interpreting and Testing the Scaling Property in Models where Inefficiency Depends on Firm Characteristics," Journal of Productivity Analysis, Springer, vol. 25(3), pages 201-212, June.
    39. repec:lan:wpaper:4487 is not listed on IDEAS
    40. repec:spr:scient:v:66:y:2006:i:2:d:10.1007_s11192-006-0028-x is not listed on IDEAS
    41. Elchanan Cohn & Samuel T. Cooper, 2004. "Multi-product Cost Functions for Universities: Economies of Scale and Scope," Chapters,in: International Handbook on the Economics of Education, chapter 15 Edward Elgar Publishing.
    42. Tommaso Agasisti & Giuseppe Catalano & Paolo Landoni & Roberto Verganti, 2012. "Evaluating the performance of academic departments: an analysis of research-related output efficiency," Research Evaluation, Oxford University Press, vol. 21(1), pages 2-14, February.
    43. Tim Coelli & Sergio Perelman, 2000. "Technical efficiency of European railways: a distance function approach," Applied Economics, Taylor & Francis Journals, vol. 32(15), pages 1967-1976.
    44. Tommaso Agasisti, 2009. "Market forces and competition in university systems: theoretical reflections and empirical evidence from Italy," International Review of Applied Economics, Taylor & Francis Journals, vol. 23(4), pages 463-483.
    45. repec:lan:wpaper:4792 is not listed on IDEAS
    46. Kao, Chiang & Hung, Hsi-Tai, 2008. "Efficiency analysis of university departments: An empirical study," Omega, Elsevier, vol. 36(4), pages 653-664, August.
    47. Smith, Jeremy & Naylor, Robin, 2001. " Determinants of Degree Performance in UK Universities: A Statistical Analysis of the 1993 Student Cohort," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 63(1), pages 29-60, February.
    48. Tiziana Laureti, 2008. "Modelling Exogenous Variables in Human Capital Formation through a Heteroscedastic Stochastic Frontier," International Advances in Economic Research, Springer;International Atlantic Economic Society, vol. 14(1), pages 76-89, February.
    49. Hung-jen Wang & Peter Schmidt, 2002. "One-Step and Two-Step Estimation of the Effects of Exogenous Variables on Technical Efficiency Levels," Journal of Productivity Analysis, Springer, vol. 18(2), pages 129-144, September.
    50. repec:spr:scient:v:96:y:2013:i:1:d:10.1007_s11192-013-0971-2 is not listed on IDEAS
    51. Charnes, A. & Cooper, W. W. & Rhodes, E., 1978. "Measuring the efficiency of decision making units," European Journal of Operational Research, Elsevier, vol. 2(6), pages 429-444, November.
    52. repec:kap:iaecre:v:14:y:2008:i:1:p:76-89 is not listed on IDEAS
    53. Huguenin, Jean-Marc, 2015. "Adjusting for the environment in DEA: A comparison of alternative models based on empirical data," Socio-Economic Planning Sciences, Elsevier, vol. 52(C), pages 41-54.
    54. Wang, Hung-Jen & Ho, Chia-Wen, 2010. "Estimating fixed-effect panel stochastic frontier models by model transformation," Journal of Econometrics, Elsevier, vol. 157(2), pages 286-296, August.
    55. Thursby, Jerry G. & Kemp, Sukanya, 2002. "Growth and productive efficiency of university intellectual property licensing," Research Policy, Elsevier, vol. 31(1), pages 109-124, January.
    56. Cullinane, Kevin & Wang, Teng-Fei & Song, Dong-Wook & Ji, Ping, 2006. "The technical efficiency of container ports: Comparing data envelopment analysis and stochastic frontier analysis," Transportation Research Part A: Policy and Practice, Elsevier, vol. 40(4), pages 354-374, May.
    57. Tommaso Agasisti, 2011. "Performances and spending efficiency in higher education: a European comparison through non-parametric approaches," Education Economics, Taylor & Francis Journals, vol. 19(2), pages 199-224.
    58. Simar, Leopold & Wilson, Paul W., 2007. "Estimation and inference in two-stage, semi-parametric models of production processes," Journal of Econometrics, Elsevier, vol. 136(1), pages 31-64, January.
    59. Roger Carrington & Tim Coelli & D. S. Prasada Rao, 2005. "The Performance Of Australian Universities: Conceptual Issues And Preliminary Results," Economic Papers, The Economic Society of Australia, vol. 24(2), pages 145-163, June.
    60. Battese, G E & Coelli, T J, 1995. "A Model for Technical Inefficiency Effects in a Stochastic Frontier Production Function for Panel Data," Empirical Economics, Springer, vol. 20(2), pages 325-332.
    61. Abbott, M. & Doucouliagos, C., 2003. "The efficiency of Australian universities: a data envelopment analysis," Economics of Education Review, Elsevier, vol. 22(1), pages 89-97, February.
    62. Aigner, Dennis & Lovell, C. A. Knox & Schmidt, Peter, 1977. "Formulation and estimation of stochastic frontier production function models," Journal of Econometrics, Elsevier, vol. 6(1), pages 21-37, July.
    63. John CUBBIN & Hadi ZAMANI, 1996. "A Comparison Of Performance Indicators For Training And Enterprise Councils In The Uk," Annals of Public and Cooperative Economics, Wiley Blackwell, vol. 67(4), pages 603-632, December.
    64. Mehdi Farsi & Massimo Filippini, 2005. "A Benchmarking Analysis of Electricity Distribution Utilities in Switzerland," CEPE Working paper series 05-43, CEPE Center for Energy Policy and Economics, ETH Zurich.
    65. Greene, William, 2005. "Reconsidering heterogeneity in panel data estimators of the stochastic frontier model," Journal of Econometrics, Elsevier, vol. 126(2), pages 269-303, June.
    66. Wilson, Paul W., 2008. "FEAR: A software package for frontier efficiency analysis with R," Socio-Economic Planning Sciences, Elsevier, vol. 42(4), pages 247-254, December.
    67. Tommaso Agasisti & Geraint Johnes, 2009. "Beyond frontiers: comparing the efficiency of higher education decision-making units across more than one country," Education Economics, Taylor & Francis Journals, vol. 17(1), pages 59-79.
    68. George E. Halkos & Nickolaos G. Tzeremes & Stavros A. Kourtzidis, 2012. "Measuring Public Owned University Departments' Efficiency: A Bootstrapped DEA Approach," Journal of Economics and Econometrics, Economics and Econometrics Society, vol. 55(2), pages 1-24.
    69. Johnes, Geraint, 1998. "The Costs of Multi-product Organizations and the Heuristic Evaluation of Industrial Structure," Socio-Economic Planning Sciences, Elsevier, vol. 32(3), pages 199-209, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gralka, Sabine, 2018. "Stochastic frontier analysis in higher education: A systematic review," CEPIE Working Papers 05/18, Technische Universität Dresden, Center of Public and International Economics (CEPIE).

    More about this item

    Keywords

    University league tables; Efficiency; Data Envelopment Analysis; Stochastic Frontier Analysis;

    JEL classification:

    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • I23 - Health, Education, and Welfare - - Education - - - Higher Education; Research Institutions
    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C67 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Input-Output Models

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:soceps:v:62:y:2018:i:c:p:104-120. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Dana Niculescu). General contact details of provider: http://www.elsevier.com/locate/seps .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.