IDEAS home Printed from https://ideas.repec.org/a/spr/joecin/v18y2020i4d10.1007_s10888-020-09449-4.html
   My bibliography  Save this article

Beyond the weights: a multicriteria approach to evaluate inequality in education

Author

Listed:
  • Giuseppe Coco

    (Università di Firenze)

  • Raffaele Lagravinese

    (Università di Bari
    University of Bari “Aldo Moro” Largo Abbazia Santa Scolastica)

  • Giuliano Resce

    (Italian National Research Council (CNR))

Abstract

This paper proposes the use of a new technique, the Stochastic Multicriteria Acceptability Analysis (SMAA), to evaluate education quality at school level out of the PISA multidimensional database. SMAA produces rankings with Monte Carlo Generation of weights to estimate the probability that each school is in a certain position of the aggregate ranking, thus avoiding any arbitrary intervention of researchers. We use the rankings in 4 waves of PISA assessment to compare SMAA outcomes with Benefit of Doubt (BoD), showing that differentiation of weights matters. Considering the whole set of feasible weights by means of SMAA, we then estimate multidimensional inequality in education, and we disentangle inequality into a ‘within’ and a ‘between’ country component, in addition to a component due to overlapping, using the multidimensional ANOGI. We find that, over time, inequality within countries has increased substantially. Overlapping among countries, particularly in the upper part of the distribution has also increased quite substantially suggesting excellence is spreading among countries.

Suggested Citation

  • Giuseppe Coco & Raffaele Lagravinese & Giuliano Resce, 2020. "Beyond the weights: a multicriteria approach to evaluate inequality in education," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 18(4), pages 469-489, December.
    • Handle: RePEc:spr:joecin:v:18:y:2020:i:4:d:10.1007_s10888-020-09449-4
    DOI: 10.1007/s10888-020-09449-4
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10888-020-09449-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10888-020-09449-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Paolo Paruolo & Michaela Saisana & Andrea Saltelli, 2013. "Ratings and rankings: voodoo or science?," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 176(3), pages 609-634, June.
    2. Antonio Villar, 2016. "Educational poverty as a welfare loss: Low performance in the OECD according to PISA 2012," Working Papers 16.04, Universidad Pablo de Olavide, Department of Economics.
    3. Francisco H. G. Ferreira & Jérémie Gignoux, 2014. "The Measurement of Educational Inequality: Achievement and Opportunity," The World Bank Economic Review, World Bank, vol. 28(2), pages 210-246.
    4. Daniele Checchi & Vito Peragine, 2010. "Inequality of opportunity in Italy," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 8(4), pages 429-450, December.
    5. Paolo Liberati, 2015. "The World Distribution of Income And Its Inequality, 1970–2009," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 61(2), pages 248-273, June.
    6. Mark Bailey & Vani Borooah, 2010. "What enhances mathematical ability? A cross-country analysis based on test scores of 15-year-olds," Applied Economics, Taylor & Francis Journals, vol. 42(29), pages 3723-3733.
    7. Tervonen, Tommi & Lahdelma, Risto, 2007. "Implementing stochastic multicriteria acceptability analysis," European Journal of Operational Research, Elsevier, vol. 178(2), pages 500-513, April.
    8. Nicholas Bloom & Renata Lemos & Raffaella Sadun & John Van Reenen, 2015. "Does Management Matter in schools?," Economic Journal, Royal Economic Society, vol. 0(584), pages 647-674, May.
    9. Foster, James & Greer, Joel & Thorbecke, Erik, 1984. "A Class of Decomposable Poverty Measures," Econometrica, Econometric Society, vol. 52(3), pages 761-766, May.
    10. Kristof De Witte & Laura López-Torres, 2017. "Efficiency in education: a review of literature and a way forward," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 68(4), pages 339-363, April.
    11. Iñaki Permanyer, 2011. "Assessing The Robustness Of Composite Indices Rankings," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 57(2), pages 306-326, June.
    12. Giannis Karagiannis & Georgia Paschalidou, 2017. "Assessing research effectiveness: a comparison of alternative nonparametric models," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 68(4), pages 456-468, April.
    13. Lovell, C. A. Knox & Pastor, Jesus T., 1999. "Radial DEA models without inputs or without outputs," European Journal of Operational Research, Elsevier, vol. 118(1), pages 46-51, October.
    14. Lahdelma, Risto & Hokkanen, Joonas & Salminen, Pekka, 1998. "SMAA - Stochastic multiobjective acceptability analysis," European Journal of Operational Research, Elsevier, vol. 106(1), pages 137-143, April.
    15. Agasisti, Tommaso & Longobardi, Sergio, 2014. "Inequality in education: Can Italian disadvantaged students close the gap?," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 52(C), pages 8-20.
    16. James Foster & Mark McGillivray & Suman Seth, 2009. "Rank Robustness of Composite Indices," OPHI Working Papers 26, Queen Elizabeth House, University of Oxford.
    17. Koen Decancq & Luc Van Ootegem & Elsy Verhofstadt, 2013. "What If We Voted on the Weights of a Multidimensional Well‐Being Index? An Illustration with Flemish Data," Fiscal Studies, Institute for Fiscal Studies, vol. 34, pages 315-332, September.
    18. Subhash C. Ray & Lei Chen, 2015. "Data Envelopment Analysis for Performance Evaluation: A Child’s Guide," Springer Books, in: Subhash C. Ray & Subal C. Kumbhakar & Pami Dua (ed.), Benchmarking for Performance Evaluation, edition 127, chapter 0, pages 75-116, Springer.
    19. Daniele Checchi, 1998. "Povertà e istruzione: alcune riflessioni e una proposta di indicatori," Politica economica, Società editrice il Mulino, issue 2, pages 245-282.
    20. Tommaso Agasisti & Giuseppe Munda & Ralph Hippe, 2019. "Measuring the efficiency of European education systems by combining Data Envelopment Analysis and Multiple-Criteria Evaluation," Journal of Productivity Analysis, Springer, vol. 51(2), pages 105-124, June.
    21. Michele Raitano & Francesco Vona, 2016. "Assessing students’ equality of opportunity in OECD countries: the role of national- and school-level policies," Applied Economics, Taylor & Francis Journals, vol. 48(33), pages 3148-3163, July.
    22. Charnes, A. & Cooper, W. W. & Rhodes, E., 1978. "Measuring the efficiency of decision making units," European Journal of Operational Research, Elsevier, vol. 2(6), pages 429-444, November.
    23. Russell G. Thompson & F. D. Singleton & Robert M. Thrall & Barton A. Smith, 1986. "Comparative Site Evaluations for Locating a High-Energy Physics Lab in Texas," Interfaces, INFORMS, vol. 16(6), pages 35-49, December.
    24. Sen, Amartya, 1997. "On Economic Inequality," OUP Catalogue, Oxford University Press, number 9780198292975, Decembrie.
    25. L Cherchye & W Moesen & N Rogge & T Van Puyenbroeck & M Saisana & A Saltelli & R Liska & S Tarantola, 2008. "Creating composite indicators with DEA and robustness analysis: the case of the Technology Achievement Index," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 59(2), pages 239-251, February.
    26. Pyatt, Graham, 1976. "On the Interpretation and Disaggregation of Gini Coefficients," Economic Journal, Royal Economic Society, vol. 86(342), pages 243-255, June.
    27. Yitzhaki, Shlomo, 1994. "Economic distance and overlapping of distributions," Journal of Econometrics, Elsevier, vol. 61(1), pages 147-159, March.
    28. Koen Decancq & María Ana Lugo, 2013. "Weights in Multidimensional Indices of Wellbeing: An Overview," Econometric Reviews, Taylor & Francis Journals, vol. 32(1), pages 7-34, January.
    29. Inglehart, Ronald F. & Norris, Pippa, 2016. "Trump, Brexit, and the Rise of Populism: Economic Have-Nots and Cultural Backlash," Working Paper Series 16-026, Harvard University, John F. Kennedy School of Government.
    30. Herrero, Carmen & Mendez, Ildefonso & Villar, Antonio, 2014. "Analysis of group performance with categorical data when agents are heterogeneous: The evaluation of scholastic performance in the OECD through PISA," Economics of Education Review, Elsevier, vol. 40(C), pages 140-151.
    31. Andrei Bougrov & Robert Johnson & Benno Ndulo & Pedro Paez & Avinash Persaud & Heidemarie Wieczorek-Zeul & Akhtar Aziz Zeti & Charles Goodhart & Jomo Kwame Sundaram & Youssef Boutros-Ghali & José Anto, 2010. "The Stiglitz Report," Working Papers hal-03415638, HAL.
    32. Kristof De Witte & Mika Kortelainen, 2013. "What explains the performance of students in a heterogeneous environment? Conditional efficiency estimation with continuous and discrete environmental variables," Applied Economics, Taylor & Francis Journals, vol. 45(17), pages 2401-2412, June.
    33. Tommaso Agasisti & Sergio Longobardi & Vincenzo Prete & Felice Russo, 2018. "Multidimensional poverty measures for analysing educational poverty in European countries," Working papers 73, Società Italiana di Economia Pubblica.
    34. Lagravinese, Raffaele & Liberati, Paolo & Resce, Giuliano, 2020. "The impact of economic, social and cultural conditions on educational attainments," Journal of Policy Modeling, Elsevier, vol. 42(1), pages 112-132.
    35. Laurens Cherchye & Willem Moesen & Nicky Rogge & Tom Puyenbroeck, 2007. "An Introduction to ‘Benefit of the Doubt’ Composite Indicators," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 82(1), pages 111-145, May.
    36. Gamboa, Luis Fernando & Waltenberg, Fábio D., 2012. "Inequality of opportunity for educational achievement in Latin America: Evidence from PISA 2006–2009," Economics of Education Review, Elsevier, vol. 31(5), pages 694-708.
    37. Risto Lahdelma & Pekka Salminen, 2001. "SMAA-2: Stochastic Multicriteria Acceptability Analysis for Group Decision Making," Operations Research, INFORMS, vol. 49(3), pages 444-454, June.
    38. Raj Chetty & Nathaniel Hendren & Lawrence F. Katz, 2016. "The Effects of Exposure to Better Neighborhoods on Children: New Evidence from the Moving to Opportunity Experiment," American Economic Review, American Economic Association, vol. 106(4), pages 855-902, April.
    39. Salvatore Greco & Alessio Ishizaka & Menelaos Tasiou & Gianpiero Torrisi, 2019. "On the Methodological Framework of Composite Indices: A Review of the Issues of Weighting, Aggregation, and Robustness," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 141(1), pages 61-94, January.
    40. Sen, Amartya K, 1976. "Poverty: An Ordinal Approach to Measurement," Econometrica, Econometric Society, vol. 44(2), pages 219-231, March.
    41. Lagravinese, Raffaele & Liberati, Paolo & Resce, Giuliano, 2019. "Exploring health outcomes by stochastic multicriteria acceptability analysis: An application to Italian regions," European Journal of Operational Research, Elsevier, vol. 274(3), pages 1168-1179.
    42. M. Saisana & A. Saltelli & S. Tarantola, 2005. "Uncertainty and sensitivity analysis techniques as tools for the quality assessment of composite indicators," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 168(2), pages 307-323, March.
    43. Silvia Angilella & Marta Bottero & Salvatore Corrente & Valentina Ferretti & Salvatore Greco & Isabella M. Lami, 2016. "Non Additive Robust Ordinal Regression for urban and territorial planning: an application for siting an urban waste landfill," Annals of Operations Research, Springer, vol. 245(1), pages 427-456, October.
    44. Richard J. Murnane & John B. Willett & Yves Duhaldeborde & John H. Tyler, 2000. "How important are the cognitive skills of teenagers in predicting subsequent earnings?," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 19(4), pages 547-568.
    45. Eric A. Hanushek & Margaret E. Raymond, 2006. "School accountability and student performance," Regional Economic Development, Federal Reserve Bank of St. Louis, issue Mar, pages 51-61.
    46. Shlomo Yitzhaki & Robert I. Lerman, 1991. "Income Stratification And Income Inequality," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 37(3), pages 313-329, September.
    47. De Witte, Kristof & Schiltz, Fritz, 2018. "Measuring and explaining organizational effectiveness of school districts: Evidence from a robust and conditional Benefit-of-the-Doubt approach," European Journal of Operational Research, Elsevier, vol. 267(3), pages 1172-1181.
    48. Salvatore Greco & Alessio Ishizaka & Benedetto Matarazzo & Gianpiero Torrisi, 2018. "Stochastic multi-attribute acceptability analysis (SMAA): an application to the ranking of Italian regions," Regional Studies, Taylor & Francis Journals, vol. 52(4), pages 585-600, April.
    49. Andrew Sharpe, 2004. "Literature Review of Frameworks for Macro-indicators," CSLS Research Reports 2004-03, Centre for the Study of Living Standards.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jos'e Alejandro Mendoza & Faustino Prieto & Jos'e Mar'ia Sarabia, 2021. "Inequality in the use frequency of patent technology codes," Papers 2111.11211, arXiv.org.
    2. Cristian Barra & Raffaele Lagravinese & Roberto Zotti, 2022. "Exploring hospital efficiency within and between Italian regions: new empirical evidence," Journal of Productivity Analysis, Springer, vol. 57(3), pages 269-284, June.
    3. Paolo Liberati & Giuliano Resce, 2022. "Regional Well-Being and its Inequality in the OECD Member Countries," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 20(3), pages 671-700, September.
    4. Gómez-Fernández, Nerea & Mediavilla, Mauro, 2021. "Exploring the relationship between Information and Communication Technologies (ICT) and academic performance: A multilevel analysis for Spain," Socio-Economic Planning Sciences, Elsevier, vol. 77(C).
    5. Manuel Sousa & Maria Fatima Almeida & Rodrigo Calili, 2021. "Multiple Criteria Decision Making for the Achievement of the UN Sustainable Development Goals: A Systematic Literature Review and a Research Agenda," Sustainability, MDPI, vol. 13(8), pages 1-37, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Greco, Salvatore & Ishizaka, Alessio & Tasiou, Menelaos & Torrisi, Gianpiero, 2019. "Sigma-Mu efficiency analysis: A methodology for evaluating units through composite indicators," European Journal of Operational Research, Elsevier, vol. 278(3), pages 942-960.
    2. Raffaele Lagravinese & Paolo Liberati & Giuliano Resce, 2020. "Measuring Health Inequality in US: A Composite Index Approach," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 147(3), pages 921-946, February.
    3. Giuliano Resce & Fritz Schiltz, 2021. "Sustainable Development in Europe: A Multicriteria Decision Analysis," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 67(2), pages 509-529, June.
    4. Ishizaka, Alessio & Resce, Giuliano, 2021. "Best-Worst PROMETHEE method for evaluating school performance in the OECD's PISA project," Socio-Economic Planning Sciences, Elsevier, vol. 73(C).
    5. Greco, Salvatore & Ishizaka, Alessio & Resce, Giuliano & Torrisi, Gianpiero, 2017. "Is the Grass Always Greener on the Other Side of the fence? Composite Index of Well-Being Taking into Account the Local Relative Appreciations in Better Life Index," MPRA Paper 82718, University Library of Munich, Germany.
    6. Salvatore Greco & Alessio Ishizaka & Menelaos Tasiou & Gianpiero Torrisi, 2019. "On the Methodological Framework of Composite Indices: A Review of the Issues of Weighting, Aggregation, and Robustness," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 141(1), pages 61-94, January.
    7. Greco, Salvatore & Ishizaka, Alessio & Tasiou, Menelaos & Torrisi, Gianpiero, 2018. "σ-µ efficiency analysis: A new methodology for evaluating units through composite indices," MPRA Paper 83569, University Library of Munich, Germany.
    8. Paolo Liberati & Giuliano Resce, 2022. "Regional Well-Being and its Inequality in the OECD Member Countries," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 20(3), pages 671-700, September.
    9. Milica Maricic & Jose A. Egea & Veljko Jeremic, 2019. "A Hybrid Enhanced Scatter Search—Composite I-Distance Indicator (eSS-CIDI) Optimization Approach for Determining Weights Within Composite Indicators," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 144(2), pages 497-537, July.
    10. Lagravinese, Raffaele & Liberati, Paolo & Resce, Giuliano, 2019. "Exploring health outcomes by stochastic multicriteria acceptability analysis: An application to Italian regions," European Journal of Operational Research, Elsevier, vol. 274(3), pages 1168-1179.
    11. Raffaele Lagravinese & Paolo Liberati & Giuliano Resce, 2017. "Exploring health outcomes by stochastic multi-objective acceptability analysis: an application to Italian regions," Working Papers. Collection B: Regional and sectoral economics 1703, Universidade de Vigo, GEN - Governance and Economics research Network.
    12. Tommaso Agasisti & Sergio Longobardi & Vincenzo Prete & Felice Russo, 2018. "Multidimensional poverty measures for analysing educational poverty in European countries," Working papers 73, Società Italiana di Economia Pubblica.
    13. Gaganis, Chrysovalantis & Pasiouras, Fotios & Tasiou, Menelaos & Zopounidis, Constantin, 2021. "CISEF: A composite index of social, environmental and financial performance," European Journal of Operational Research, Elsevier, vol. 291(1), pages 394-409.
    14. Greco, Salvatore & Ishizaka, Alessio & Tasiou, Menelaos & Torrisi, Gianpiero, 2019. "The Ordinal Input for Cardinal Output Approach of Non-compensatory Composite Indicators: The PROMETHEE Scoring Method," MPRA Paper 95816, University Library of Munich, Germany.
    15. Greco, Salvatore & Ishizaka, Alessio & Tasiou, Menelaos & Torrisi, Gianpiero, 2021. "The ordinal input for cardinal output approach of non-compensatory composite indicators: the PROMETHEE scoring method," European Journal of Operational Research, Elsevier, vol. 288(1), pages 225-246.
    16. De Witte, Kristof & Schiltz, Fritz, 2018. "Measuring and explaining organizational effectiveness of school districts: Evidence from a robust and conditional Benefit-of-the-Doubt approach," European Journal of Operational Research, Elsevier, vol. 267(3), pages 1172-1181.
    17. Fusco, Elisa & Vidoli, Francesco & Rogge, Nicky, 2020. "Spatial directional robust Benefit of the Doubt approach in presence of undesirable output: An application to Italian waste sector," Omega, Elsevier, vol. 94(C).
    18. Su, Weihua & Chen, Sibo & Zhang, Chonghui & Li, Kevin W., 2023. "A subgroup dominance-based benefit of the doubt method for addressing rank reversals: A case study of the human development index in Europe," European Journal of Operational Research, Elsevier, vol. 307(3), pages 1299-1317.
    19. Paolo Liberati & Raffaele Lagravinese & Giuliano Resce, 2017. "How Does Economic Social And Cultural Status Affect The Efficiency Of Educational Attainments? A Comparative Analysis On Pisa Results," Departmental Working Papers of Economics - University 'Roma Tre' 0217, Department of Economics - University Roma Tre.
    20. Greco, Salvatore & Ishizaka, Alessio & Matarazzo, Benedetto & Torrisi, Gianpiero, 2015. "Stochastic Multiattribute Acceptability Analysis: an application to the ranking of Italian regions," MPRA Paper 68508, University Library of Munich, Germany.

    More about this item

    Keywords

    Education inequality; PISA; SMAA; ANOGI; Anywhere and somewhere;
    All these keywords.

    JEL classification:

    • I14 - Health, Education, and Welfare - - Health - - - Health and Inequality
    • C44 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - Operations Research; Statistical Decision Theory

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:joecin:v:18:y:2020:i:4:d:10.1007_s10888-020-09449-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.