IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v289y2021i3p1188-1208.html
   My bibliography  Save this article

Performance assessment of upper secondary schools in Italian regions using a circular pseudo-Malmquist index

Author

Listed:
  • Camanho, Ana S.
  • Varriale, Luisa
  • Barbosa, Flávia
  • Sobral, Thiago

Abstract

This paper investigates the relationship between students’ performance and the type of school attended during upper secondary education. The performance of three different types of schools (Liceo, Technical and Professional schools) in four Italian macroregions (North West, North East, Centre, South & Islands) is investigated. A benchmarking analysis of the variability in students’ performance among regions (within macroregions) for cohorts of students attending Liceo is also conducted. The data was collected at the student level from the Italian Institute for the Evaluation of Education System (INVALSI), for the academic year 2017/18. Families with higher socio-economic status may self-select into Liceo, so a direct comparison with vocational schools could lead to biased conclusions regarding the impact of school type on student performance. To overcome this limitation, we used a Propensity Score Matching approach prior to the estimation of efficiency. A pseudo-Malmquist index, based on a metafrontier and satisfying the circular property, is developed. It enables comparing the location of the best-practice frontier for each type of school and the spread in the educational efficiency of the students attending each type of school. Thus, best performance of a given school type corresponds to the combined effect of these two aspects. This study is an interesting starting point to challenge the stereotypes that persist in Italy, especially concerning general and vocational studies and geographic differences in educational achievements.

Suggested Citation

  • Camanho, Ana S. & Varriale, Luisa & Barbosa, Flávia & Sobral, Thiago, 2021. "Performance assessment of upper secondary schools in Italian regions using a circular pseudo-Malmquist index," European Journal of Operational Research, Elsevier, vol. 289(3), pages 1188-1208.
  • Handle: RePEc:eee:ejores:v:289:y:2021:i:3:p:1188-1208
    DOI: 10.1016/j.ejor.2020.07.050
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221720306767
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ejor.2020.07.050?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Daniele Checchi & Luca Flabbi, 2013. "Intergenerational Mobility and Schooling Decisions in Germany and Italy: The Impact of Secondary School Tracks," Rivista di Politica Economica, SIPI Spa, issue 3, pages 7-57, July-Sept.
    2. Orazio Giancola & Luca Salmieri, 2020. "Family Background, School-Track and Macro-Area: the Complex Chains of Education Inequalities in Italy," Working Papers 4/20, Sapienza University of Rome, DISS.
    3. Adriana Di Liberto & Fabiano Schivardi & Giovanni Sulis, 2015. "Managerial practices and student performance," Economic Policy, CEPR, CESifo, Sciences Po;CES;MSH, vol. 30(84), pages 683-728.
    4. Pastor, Jesus T. & Lovell, C.A. Knox, 2005. "A global Malmquist productivity index," Economics Letters, Elsevier, vol. 88(2), pages 266-271, August.
    5. Daniele Checchi & Vito Peragine, 2010. "Inequality of opportunity in Italy," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 8(4), pages 429-450, December.
    6. José Manuel Cordero Ferrera & Eva Crespo Cebada & Francisco Pedraja Chaparro & Daniel Santín González, 2011. "Exploring Educational Efficiency Divergences Across Spanish Regions In Pisa 2006," Revista de Economia Aplicada, Universidad de Zaragoza, Departamento de Estructura Economica y Economia Publica, vol. 19(3), pages 117-145, Winter.
    7. Renata Lemos, 2014. "In brief: Does management matter in schools?," CentrePiece - The magazine for economic performance 424, Centre for Economic Performance, LSE.
    8. Nicholas Bloom & Renata Lemos & Raffaella Sadun & John Van Reenen, 2015. "Does Management Matter in schools?," Economic Journal, Royal Economic Society, vol. 0(584), pages 647-674, May.
    9. Kristof De Witte & Laura López-Torres, 2017. "Efficiency in education: a review of literature and a way forward," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 68(4), pages 339-363, April.
    10. Checchi, Daniele & Ichino, Andrea & Rustichini, Aldo, 1999. "More equal but less mobile?: Education financing and intergenerational mobility in Italy and in the US," Journal of Public Economics, Elsevier, vol. 74(3), pages 351-393, December.
    11. M C A S Portela & A S Camanho, 2010. "Analysis of complementary methodologies for the estimation of school value added," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 61(7), pages 1122-1132, July.
    12. Tanja Kirjavainen, 2012. "Efficiency of Finnish general upper secondary schools: an application of stochastic frontier analysis with panel data," Education Economics, Taylor & Francis Journals, vol. 20(4), pages 343-364, April.
    13. C. Masci & F. Ieva & T. Agasisti & A. M. Paganoni, 2017. "Bivariate multilevel models for the analysis of mathematics and reading pupils' achievements," Journal of Applied Statistics, Taylor & Francis Journals, vol. 44(7), pages 1296-1317, May.
    14. W. Alexander & Alfred Haug & Mohammad Jaforullah, 2010. "A two-stage double-bootstrap data envelopment analysis of efficiency differences of New Zealand secondary schools," Journal of Productivity Analysis, Springer, vol. 34(2), pages 99-110, October.
    15. Eric A. Hanushek, 1979. "Conceptual and Empirical Issues in the Estimation of Educational Production Functions," Journal of Human Resources, University of Wisconsin Press, vol. 14(3), pages 351-388.
    16. Masci, Chiara & De Witte, Kristof & Agasisti, Tommaso, 2018. "The influence of school size, principal characteristics and school management practices on educational performance: An efficiency analysis of Italian students attending middle schools," Socio-Economic Planning Sciences, Elsevier, vol. 61(C), pages 52-69.
    17. Eva Crespo-Cebada & Francisco Pedraja-Chaparro & Daniel Santín, 2014. "Does school ownership matter? An unbiased efficiency comparison for regions of Spain," Journal of Productivity Analysis, Springer, vol. 41(1), pages 153-172, February.
    18. Charnes, A. & Cooper, W. W. & Rhodes, E., 1978. "Measuring the efficiency of decision making units," European Journal of Operational Research, Elsevier, vol. 2(6), pages 429-444, November.
    19. Giorgio Brunello & Daniele Checchi, 2007. "Does school tracking affect equality of opportunity? New international evidence [‘Educational opportunities and the role of institutions’]," Economic Policy, CEPR, CESifo, Sciences Po;CES;MSH, vol. 22(52), pages 782-861.
    20. Agasisti, Tommaso & Cordero-Ferrera, Jose M., 2013. "Educational disparities across regions: A multilevel analysis for Italy and Spain," Journal of Policy Modeling, Elsevier, vol. 35(6), pages 1079-1102.
    21. A. Camanho & R. Dyson, 2006. "Data envelopment analysis and Malmquist indices for measuring group performance," Journal of Productivity Analysis, Springer, vol. 26(1), pages 35-49, August.
    22. Susanna Loeb & Demetra Kalogrides & Tara Béteille, 2012. "Effective Schools: Teacher Hiring, Assignment, Development, and Retention," Education Finance and Policy, MIT Press, vol. 7(3), pages 269-304, July.
    23. M-J Mancebón & J Calero & Á Choi & D P Ximénez-de-Embún, 2012. "The efficiency of public and publicly subsidized high schools in Spain: Evidence from PISA-2006," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 63(11), pages 1516-1533, November.
    24. Aparicio, Juan & Crespo-Cebada, Eva & Pedraja-Chaparro, Francisco & Santín, Daniel, 2017. "Comparing school ownership performance using a pseudo-panel database: A Malmquist-type index approach," European Journal of Operational Research, Elsevier, vol. 256(2), pages 533-542.
    25. Giuseppe Di Giacomo & Aline Pennisi, 2015. "Assessing Primary and Lower Secondary School Efficiency Within Northern, Central and Southern Italy," Italian Economic Journal: A Continuation of Rivista Italiana degli Economisti and Giornale degli Economisti, Springer;Società Italiana degli Economisti (Italian Economic Association), vol. 1(2), pages 287-311, July.
    26. Christopher O’Donnell & D. Rao & George Battese, 2008. "Metafrontier frameworks for the study of firm-level efficiencies and technology ratios," Empirical Economics, Springer, vol. 34(2), pages 231-255, March.
    27. Ichino, Andrea & Tabellini, Guido, 2014. "Freeing the Italian school system," Labour Economics, Elsevier, vol. 30(C), pages 113-128.
    28. Gregory F. Branch & Eric A. Hanushek & Steven G. Rivkin, 2012. "Estimating the Effect of Leaders on Public Sector Productivity: The Case of School Principals," NBER Working Papers 17803, National Bureau of Economic Research, Inc.
    29. Luigi Benfratello & Giuseppe Sorrenti & Gilberto Turati, 2015. "Tracking in the Tracks in the Italian Schooling: Inequality Patterns in an Urban Context," Working papers 030, Department of Economics and Statistics (Dipartimento di Scienze Economico-Sociali e Matematico-Statistiche), University of Torino.
    30. Masci, Chiara & Ieva, Francesca & Agasisti, Tommaso & Paganoni, Anna Maria, 2016. "Does class matter more than school? Evidence from a multilevel statistical analysis on Italian junior secondary school students," Socio-Economic Planning Sciences, Elsevier, vol. 54(C), pages 47-57.
    31. Hanushek, Eric A. & Luque, Javier A., 2003. "Efficiency and equity in schools around the world," Economics of Education Review, Elsevier, vol. 22(5), pages 481-502, October.
    32. Aparicio, Juan & Santin, Daniel, 2018. "A note on measuring group performance over time with pseudo-panels," European Journal of Operational Research, Elsevier, vol. 267(1), pages 227-235.
    33. Agasisti, Tommaso, 2011. "How competition affects schools' performances: Does specification matter?," Economics Letters, Elsevier, vol. 110(3), pages 259-261, March.
    34. Gronberg, Timothy J. & Jansen, Dennis W. & Taylor, Lori L., 2012. "The relative efficiency of charter schools: A cost frontier approach," Economics of Education Review, Elsevier, vol. 31(2), pages 302-317.
    35. Eric A. Hanushek, 2003. "The Failure of Input-Based Schooling Policies," Economic Journal, Royal Economic Society, vol. 113(485), pages 64-98, February.
    36. Afsharian, Mohsen & Ahn, Heinz & Harms, Sören Guntram, 2019. "Performance comparison of management groups under centralised management," European Journal of Operational Research, Elsevier, vol. 278(3), pages 845-854.
    37. George E. Battese & D. S. Prasada Rao, 2002. "Technology Gap, Efficiency, and a Stochastic Metafrontier Function," International Journal of Business and Economics, School of Management Development, Feng Chia University, Taichung, Taiwan, vol. 1(2), pages 87-93, August.
    38. Claudio Quintano & Rosalia Castellano & Sergio Longobardi, 2009. "L'influenza dei fattori socio economici sulle competenze degli studenti italiani. Un'analisi multilevel dei dati PISA 2006," RIVISTA DI ECONOMIA E STATISTICA DEL TERRITORIO, FrancoAngeli Editore, vol. 2009(2), pages 109-149.
    39. Giambona, Francesca & Porcu, Mariano, 2018. "School size and students' achievement. Empirical evidences from PISA survey data," Socio-Economic Planning Sciences, Elsevier, vol. 64(C), pages 66-77.
    40. Masci, Chiara & Johnes, Geraint & Agasisti, Tommaso, 2018. "Student and school performance across countries: A machine learning approach," European Journal of Operational Research, Elsevier, vol. 269(3), pages 1072-1085.
    41. M C S Portela & A S Camanho & D Borges, 2012. "Performance assessment of secondary schools: the snapshot of a country taken by DEA," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 63(8), pages 1098-1115, August.
    42. R. D. Banker & A. Charnes & W. W. Cooper, 1984. "Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis," Management Science, INFORMS, vol. 30(9), pages 1078-1092, September.
    43. Tommaso Agasisti & Patrizia Falzetti & Mara Soncin, 2016. "Italian school principals’ managerial behaviors and students’ test scores: an empirical analysis," Working papers 43, Società Italiana di Economia Pubblica.
    44. Tommaso Agasisti, 2013. "The efficiency of Italian secondary schools and the potential role of competition: a data envelopment analysis using OECD-PISA2006 data," Education Economics, Taylor & Francis Journals, vol. 21(5), pages 520-544, December.
    45. Thieme, Claudio & Prior, Diego & Tortosa-Ausina, Emili, 2013. "A multilevel decomposition of school performance using robust nonparametric frontier techniques," Economics of Education Review, Elsevier, vol. 32(C), pages 104-121.
    46. Carlos D. Mayen & Joseph V. Balagtas & Corinne E. Alexander, 2010. "Technology Adoption and Technical Efficiency: Organic and Conventional Dairy Farms in the United States," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 92(1), pages 181-195.
    47. Mariagiulia Matteucci & Stefania Mignani, 2014. "Exploring Regional Differences in the Reading Competencies of Italian Students," Evaluation Review, , vol. 38(3), pages 251-290, June.
    48. Podinovski, Victor V. & Ismail, Ihsan & Bouzdine-Chameeva, Tatiana & Zhang, Wenjuan, 2014. "Combining the assumptions of variable and constant returns to scale in the efficiency evaluation of secondary schools," European Journal of Operational Research, Elsevier, vol. 239(2), pages 504-513.
    49. Giambona, Francesca & Porcu, Mariano, 2015. "Student background determinants of reading achievement in Italy. A quantile regression analysis," International Journal of Educational Development, Elsevier, vol. 44(C), pages 95-107.
    50. Jason A. Grissom & Demetra Kalogrides & Susanna Loeb, 2012. "Using Student Test Scores to Measure Principal Performance," NBER Working Papers 18568, National Bureau of Economic Research, Inc.
    51. De Witte, Kristof & Schiltz, Fritz, 2018. "Measuring and explaining organizational effectiveness of school districts: Evidence from a robust and conditional Benefit-of-the-Doubt approach," European Journal of Operational Research, Elsevier, vol. 267(3), pages 1172-1181.
    52. Guarini, Giulio & Laureti, Tiziana & Garofalo, Giuseppe, 2018. "Territorial and individual educational inequality: A Capability Approach analysis for Italy," Economic Modelling, Elsevier, vol. 71(C), pages 247-262.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Tourinho, Marco & Barbosa, Flávia & Santos, Paulo Rosa & Pinto, Francisco Taveira & Camanho, Ana S., 2023. "Productivity change in Brazilian water services: A benchmarking study of national and regional trends," Socio-Economic Planning Sciences, Elsevier, vol. 86(C).
    2. Chen, Xiang & Grifell-Tatjé, Emili & Fu, Tsu-Tan, 2023. "A profit difference decomposition model for measuring group performance: an application to Chinese and Taiwanese commercial banks," Omega, Elsevier, vol. 120(C).
    3. Camanho, Ana S. & Stumbriene, Dovile & Barbosa, Flávia & Jakaitiene, Audrone, 2023. "The assessment of performance trends and convergence in education and training systems of European countries," European Journal of Operational Research, Elsevier, vol. 305(1), pages 356-372.
    4. Aparicio, Juan & Santín, Daniel, 2022. "A New Malmquist Index Based on a Standard Technology for Measuring Total Factor Productivity Changes," MPRA Paper 114467, University Library of Munich, Germany.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kristof De Witte & Laura López-Torres, 2017. "Efficiency in education: a review of literature and a way forward," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 68(4), pages 339-363, April.
    2. Aparicio, Juan & Ortiz, Lidia & Santín, Daniel, 2021. "Comparing group performance over time through the Luenberger productivity indicator: An application to school ownership in European countries," European Journal of Operational Research, Elsevier, vol. 294(2), pages 651-672.
    3. Paolo Liberati & Raffaele Lagravinese & Giuliano Resce, 2017. "How Does Economic Social And Cultural Status Affect The Efficiency Of Educational Attainments? A Comparative Analysis On Pisa Results," Departmental Working Papers of Economics - University 'Roma Tre' 0217, Department of Economics - University Roma Tre.
    4. K. Kounetas & G. Androulakis & M. Kaisari & G. Manousakis, 2023. "Educational reforms and secondary school's efficiency performance in Greece: a bootstrap DEA and multilevel approach," Operational Research, Springer, vol. 23(1), pages 1-29, March.
    5. Alexander Arévalo S & Víctor Giménez G & Diego Prior J, 2022. "Análisis de eficiencia en educación: una aplicación del método StoNED," Revista Desarrollo y Sociedad, Universidad de los Andes,Facultad de Economía, CEDE, vol. 92(2), pages 45-91, October.
    6. Lagravinese, Raffaele & Liberati, Paolo & Resce, Giuliano, 2020. "The impact of economic, social and cultural conditions on educational attainments," Journal of Policy Modeling, Elsevier, vol. 42(1), pages 112-132.
    7. Paola Azar & Gabriela Sicilia, 2021. "An assessment of pupil and school performance in public primary education in Uruguay," Documentos de Trabajo (working papers) 21-22, Instituto de Economía - IECON.
    8. Azar, Paola & Sicilia, Gabriela, 2023. "Assessing potential improvement in education: Unravelling pupils' and schools’ inefficiencies using a multi-level metafrontier approach," Economic Modelling, Elsevier, vol. 118(C).
    9. Johnes, Jill, 2015. "Operational Research in education," European Journal of Operational Research, Elsevier, vol. 243(3), pages 683-696.
    10. Chen, Xiang & Grifell-Tatjé, Emili & Fu, Tsu-Tan, 2023. "A profit difference decomposition model for measuring group performance: an application to Chinese and Taiwanese commercial banks," Omega, Elsevier, vol. 120(C).
    11. Camanho, Ana S. & Stumbriene, Dovile & Barbosa, Flávia & Jakaitiene, Audrone, 2023. "The assessment of performance trends and convergence in education and training systems of European countries," European Journal of Operational Research, Elsevier, vol. 305(1), pages 356-372.
    12. López-Torres, Laura & Nicolini, Rosella & Prior, Diego, 2017. "Does strategic interaction affect demand for school places? A conditional efficiency approach," Regional Science and Urban Economics, Elsevier, vol. 65(C), pages 89-103.
    13. Cordero, José Manuel & Prior, Diego & Simancas Rodríguez, Rosa, 2013. "A comparison of public and private schools in Spain using robust nonparametric frontier methods," MPRA Paper 51375, University Library of Munich, Germany.
    14. Arbona, Alexei & Giménez, Víctor & López-Estrada, Sebastián & Prior, Diego, 2022. "Efficiency and quality in Colombian education: An application of the metafrontier Malmquist-Luenberger productivity index," Socio-Economic Planning Sciences, Elsevier, vol. 79(C).
    15. D’Inverno, Giovanna & Smet, Mike & De Witte, Kristof, 2021. "Impact evaluation in a multi-input multi-output setting: Evidence on the effect of additional resources for schools," European Journal of Operational Research, Elsevier, vol. 290(3), pages 1111-1124.
    16. Afsharian, Mohsen & Ahn, Heinz & Harms, Sören Guntram, 2019. "Performance comparison of management groups under centralised management," European Journal of Operational Research, Elsevier, vol. 278(3), pages 845-854.
    17. Aparicio, Juan & Cordero, Jose M. & Gonzalez, Martin & Lopez-Espin, Jose J., 2018. "Using non-radial DEA to assess school efficiency in a cross-country perspective: An empirical analysis of OECD countries," Omega, Elsevier, vol. 79(C), pages 9-20.
    18. Aparicio, Juan & Crespo-Cebada, Eva & Pedraja-Chaparro, Francisco & Santín, Daniel, 2017. "Comparing school ownership performance using a pseudo-panel database: A Malmquist-type index approach," European Journal of Operational Research, Elsevier, vol. 256(2), pages 533-542.
    19. Gabriella Sicilia & Rosa Simancas, 2023. "Eficiencia y equidad educativa en España: un análisis comparativo a nivel regional," Hacienda Pública Española / Review of Public Economics, IEF, vol. 245(2), pages 7-33, June.
    20. López-Torres, Laura & Prior, Diego, 2022. "Long-term efficiency of public service provision in a context of budget restrictions. An application to the education sector," Socio-Economic Planning Sciences, Elsevier, vol. 81(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:289:y:2021:i:3:p:1188-1208. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.