IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v267y2018i3p1172-1181.html
   My bibliography  Save this article

Measuring and explaining organizational effectiveness of school districts: Evidence from a robust and conditional Benefit-of-the-Doubt approach

Author

Listed:
  • De Witte, Kristof
  • Schiltz, Fritz

Abstract

This paper constructs an innovative approach to measure school district organizational effectiveness. We use the fully nonparametric Benefit-of-the-Doubt approach to construct a composite indicator which reduces subjectivity in choosing weights attached to different functions of school districts. The model is extended to reduce the influence of outliers (robust) and to account for the exogenous environment (conditional). We apply the suggested technique to unique and self-collected data from surveys and interviews taken from school board members and principals. We illustrate why accounting for respondent characteristics is crucial when evaluating school districts. Next, we analyze the obtained aggregate measures to identify which organizational structures, board characteristics and management styles are of importance for the organizational effectiveness of school districts.

Suggested Citation

  • De Witte, Kristof & Schiltz, Fritz, 2018. "Measuring and explaining organizational effectiveness of school districts: Evidence from a robust and conditional Benefit-of-the-Doubt approach," European Journal of Operational Research, Elsevier, vol. 267(3), pages 1172-1181.
    • Handle: RePEc:eee:ejores:v:267:y:2018:i:3:p:1172-1181
    DOI: 10.1016/j.ejor.2017.12.034
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221717311682
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ejor.2017.12.034?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. De Witte, Kristof & Geys, Benny, 2013. "Citizen coproduction and efficient public good provision: Theory and evidence from local public libraries," European Journal of Operational Research, Elsevier, vol. 224(3), pages 592-602.
    2. Li, Degui & Simar, Léopold & Zelenyuk, Valentin, 2016. "Generalized nonparametric smoothing with mixed discrete and continuous data," Computational Statistics & Data Analysis, Elsevier, vol. 100(C), pages 424-444.
    3. Shwartz, Michael & Burgess, James F. & Zhu, Joe, 2016. "A DEA based composite measure of quality and its associated data uncertainty interval for health care provider profiling and pay-for-performance," European Journal of Operational Research, Elsevier, vol. 253(2), pages 489-502.
    4. Bădin, Luiza & Daraio, Cinzia & Simar, Léopold, 2012. "How to measure the impact of environmental factors in a nonparametric production model," European Journal of Operational Research, Elsevier, vol. 223(3), pages 818-833.
    5. Adriana Di Liberto & Fabiano Schivardi & Giovanni Sulis, 2015. "Managerial practices and student performance," Economic Policy, CEPR, CESifo, Sciences Po;CES;MSH, vol. 30(84), pages 683-728.
    6. Nicholas Bloom & Renata Lemos & Raffaella Sadun & John Van Reenen, 2015. "Does Management Matter in schools?," Economic Journal, Royal Economic Society, vol. 0(584), pages 647-674, May.
    7. Kristof De Witte & Laura López-Torres, 2017. "Efficiency in education: a review of literature and a way forward," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 68(4), pages 339-363, April.
    8. Alois Kneip & Léopold Simar & Paul W. Wilson, 2016. "Testing Hypotheses in Nonparametric Models of Production," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(3), pages 435-456, July.
    9. Lovell, C. A. Knox & Pastor, Jesus T., 1999. "Radial DEA models without inputs or without outputs," European Journal of Operational Research, Elsevier, vol. 118(1), pages 46-51, October.
    10. Cinzia Daraio & Léopold Simar, 2005. "Introducing Environmental Variables in Nonparametric Frontier Models: a Probabilistic Approach," Journal of Productivity Analysis, Springer, vol. 24(1), pages 93-121, September.
    11. Leach, John & Payne, A. Abigail & Chan, Steve, 2010. "The effects of school board consolidation and financing on student performance," Economics of Education Review, Elsevier, vol. 29(6), pages 1034-1046, December.
    12. Journal of Economics Bibliography, 2016. "New Economics Journals," Journal of Economics Bibliography, KSP Journals, vol. 3(1), pages 157-171, March.
    13. Verschelde, Marijn & Rogge, Nicky, 2012. "An environment-adjusted evaluation of citizen satisfaction with local police effectiveness: Evidence from a conditional Data Envelopment Analysis approach," European Journal of Operational Research, Elsevier, vol. 223(1), pages 214-225.
    14. Journal of Economics Bibliography, 2016. "New Economics Journals," Journal of Economics Bibliography, KSP Journals, vol. 3(2), pages 370-379, June.
    15. Badin, Luiza & Daraio, Cinzia & Simar, Léopold, 2010. "Optimal bandwidth selection for conditional efficiency measures: A data-driven approach," European Journal of Operational Research, Elsevier, vol. 201(2), pages 633-640, March.
    16. Johnes, Geraint & Johnes, Jill, 2009. "Higher education institutions' costs and efficiency: Taking the decomposition a further step," Economics of Education Review, Elsevier, vol. 28(1), pages 107-113, February.
    17. Charnes, A. & Cooper, W. W. & Rhodes, E., 1978. "Measuring the efficiency of decision making units," European Journal of Operational Research, Elsevier, vol. 2(6), pages 429-444, November.
    18. Daraio, Cinzia & Simar, Leopold, 2006. "A robust nonparametric approach to evaluate and explain the performance of mutual funds," European Journal of Operational Research, Elsevier, vol. 175(1), pages 516-542, November.
    19. José M. Alonso & Judith Clifton & Daniel Díaz-Fuentes, 2015. "Did New Public Management Matter? An empirical analysis of the outsourcing and decentralization effects on public sector size," Public Management Review, Taylor & Francis Journals, vol. 17(5), pages 643-660, May.
    20. Simar, Leopold & Wilson, Paul W., 2007. "Estimation and inference in two-stage, semi-parametric models of production processes," Journal of Econometrics, Elsevier, vol. 136(1), pages 31-64, January.
    21. Léopold Simar & Paul Wilson, 2011. "Two-stage DEA: caveat emptor," Journal of Productivity Analysis, Springer, vol. 36(2), pages 205-218, October.
    22. Rolf Färe & Shawna Grosskopf & William L. Weber, 1989. "Measuring School District Performance," Public Finance Review, , vol. 17(4), pages 409-428, October.
    23. R. Mark Gritz & Neil D. Theobald, 1996. "The Effects of School District Spending Priorities on Length of Stay in Teaching," Journal of Human Resources, University of Wisconsin Press, vol. 31(3), pages 477-512.
    24. Cinzia Daraio & Léopold Simar, 2007. "Conditional nonparametric frontier models for convex and nonconvex technologies: a unifying approach," Journal of Productivity Analysis, Springer, vol. 28(1), pages 13-32, October.
    25. Unnever, James D. & Kerckhoff, Allan C. & Robinson, Timothy J., 2000. "District variations in educational resources and student outcomes," Economics of Education Review, Elsevier, vol. 19(3), pages 245-259, June.
    26. Peter Hall & Jeff Racine & Qi Li, 2004. "Cross-Validation and the Estimation of Conditional Probability Densities," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 1015-1026, December.
    27. Ya Chen & Wade D. Cook & Juan Du & Hanhui Hu & Joe Zhu, 2017. "Bounded and discrete data and Likert scales in data envelopment analysis: application to regional energy efficiency in China," Annals of Operations Research, Springer, vol. 255(1), pages 347-366, August.
    28. Zanella, Andreia & Camanho, Ana S. & Dias, Teresa G., 2015. "Undesirable outputs and weighting schemes in composite indicators based on data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 245(2), pages 517-530.
    29. Kristof De Witte & Mika Kortelainen, 2013. "What explains the performance of students in a heterogeneous environment? Conditional efficiency estimation with continuous and discrete environmental variables," Applied Economics, Taylor & Francis Journals, vol. 45(17), pages 2401-2412, June.
    30. Cordero, José Manuel & Alonso-Morán, Edurne & Nuño-Solinis, Roberto & Orueta, Juan F. & Arce, Regina Sauto, 2015. "Efficiency assessment of primary care providers: A conditional nonparametric approach," European Journal of Operational Research, Elsevier, vol. 240(1), pages 235-244.
    31. Laurens Cherchye & Willem Moesen & Nicky Rogge & Tom Puyenbroeck, 2007. "An Introduction to ‘Benefit of the Doubt’ Composite Indicators," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 82(1), pages 111-145, May.
    32. Rose, Heather & Sonstelie, Jon, 2010. "School board politics, school district size, and the bargaining power of teachers' unions," Journal of Urban Economics, Elsevier, vol. 67(3), pages 438-450, May.
    33. Darold Barnum & Jason Coupet & John Gleason & Abagail McWilliams & Annaleena Parhankangas, 2017. "Impact of input substitution and output transformation on data envelopment analysis decisions," Applied Economics, Taylor & Francis Journals, vol. 49(15), pages 1543-1556, March.
    34. Andrews, Matthew & Duncombe, William & Yinger, John, 2002. "Revisiting economies of size in American education: are we any closer to a consensus?," Economics of Education Review, Elsevier, vol. 21(3), pages 245-262, June.
    35. Sung Li & Long Zhao, 2015. "The competitiveness and development strategies of provinces in China: a data envelopment analysis approach," Journal of Productivity Analysis, Springer, vol. 44(3), pages 293-307, December.
    36. Antonio García-Romero & Daniel Santín & Gabriela Sicilia, 2016. "Another brick in the wall: a new ranking of academic journals in Economics using FDH," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(1), pages 91-101, April.
    37. Cordero, José Manuel & Salinas-Jiménez, Javier & Salinas-Jiménez, M Mar, 2017. "Exploring factors affecting the level of happiness across countries: A conditional robust nonparametric frontier analysis," European Journal of Operational Research, Elsevier, vol. 256(2), pages 663-672.
    38. Dyson, R. G. & Allen, R. & Camanho, A. S. & Podinovski, V. V. & Sarrico, C. S. & Shale, E. A., 2001. "Pitfalls and protocols in DEA," European Journal of Operational Research, Elsevier, vol. 132(2), pages 245-259, July.
    39. Cazals, Catherine & Florens, Jean-Pierre & Simar, Leopold, 2002. "Nonparametric frontier estimation: a robust approach," Journal of Econometrics, Elsevier, vol. 106(1), pages 1-25, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Dr. Nuwatuhaire Benard & Dr. Rwabutogo Zogeye Marcel & Janan Mubehamwe & Dr. Phanual Murenzi, 2023. "Assessing the Relationship between Female Head Teachers’ Supportive Leadership Behaviours and Organizational Effectiveness of Secondary Schools in Rwanda, Nyagatare District," International Journal of Research and Innovation in Social Science, International Journal of Research and Innovation in Social Science (IJRISS), vol. 7(11), pages 1292-1306, November.
    2. Su, Weihua & Chen, Sibo & Zhang, Chonghui & Li, Kevin W., 2023. "A subgroup dominance-based benefit of the doubt method for addressing rank reversals: A case study of the human development index in Europe," European Journal of Operational Research, Elsevier, vol. 307(3), pages 1299-1317.
    3. D’Inverno, Giovanna & De Witte, Kristof, 2020. "Service level provision in municipalities: A flexible directional distance composite indicator," European Journal of Operational Research, Elsevier, vol. 286(3), pages 1129-1141.
    4. Stumbrienė, Dovilė & Želvys, Rimantas & Žilinskas, Julius & Dukynaitė, Rita & Jakaitienė, Audronė, 2022. "Efficiency and effectiveness analysis based on educational inclusion and fairness of European countries," Socio-Economic Planning Sciences, Elsevier, vol. 82(PB).
    5. Vilarinho, Hermilio & D’Inverno, Giovanna & Nóvoa, Henriqueta & Camanho, Ana S., 2023. "The measurement of asset management performance of water companies," Socio-Economic Planning Sciences, Elsevier, vol. 87(PA).
    6. Tommaso Agasisti & Ekaterina Shibanova, 2020. "Autonomy, Performance And Efficiency: An Empirical Analysis Of Russian Universities 2014-2018," HSE Working papers WP BRP 224/EC/2020, National Research University Higher School of Economics.
    7. Ishizaka, Alessio & Resce, Giuliano, 2021. "Best-Worst PROMETHEE method for evaluating school performance in the OECD's PISA project," Socio-Economic Planning Sciences, Elsevier, vol. 73(C).
    8. Pereira, Miguel Alves & Vilarinho, Hermilio & D’Inverno, Giovanna & Camanho, Ana S., 2023. "A regulatory robust conditional approach to measuring the efficiency of wholesale water supply and wastewater treatment services," Utilities Policy, Elsevier, vol. 83(C).
    9. Camanho, Ana S. & Stumbriene, Dovile & Barbosa, Flávia & Jakaitiene, Audrone, 2023. "The assessment of performance trends and convergence in education and training systems of European countries," European Journal of Operational Research, Elsevier, vol. 305(1), pages 356-372.
    10. Carina Steckenleiter & Michael Lechner & Tim Pawlowski & Ute Schüttoff, 2023. "Do local expenditures on sports facilities affect sports participation?," Economic Inquiry, Western Economic Association International, vol. 61(4), pages 1103-1128, October.
    11. D’Inverno, Giovanna & Carosi, Laura & Romano, Giulia, 2021. "Environmental sustainability and service quality beyond economic and financial indicators: A performance evaluation of Italian water utilities," Socio-Economic Planning Sciences, Elsevier, vol. 75(C).
    12. Alda A. Henriques & Milton Fontes & Ana S. Camanho & Giovanna D’Inverno & Pedro Amorim & Jaime Gabriel Silva, 2022. "Performance evaluation of problematic samples: a robust nonparametric approach for wastewater treatment plants," Annals of Operations Research, Springer, vol. 315(1), pages 193-220, August.
    13. Giuseppe Coco & Raffaele Lagravinese & Giuliano Resce, 2020. "Beyond the weights: a multicriteria approach to evaluate inequality in education," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 18(4), pages 469-489, December.
    14. Camanho, Ana S. & Varriale, Luisa & Barbosa, Flávia & Sobral, Thiago, 2021. "Performance assessment of upper secondary schools in Italian regions using a circular pseudo-Malmquist index," European Journal of Operational Research, Elsevier, vol. 289(3), pages 1188-1208.
    15. López-Torres, Laura & Johnes, Jill & Elliott, Caroline & Polo, Cristina, 2021. "The effects of competition and collaboration on efficiency in the UK independent school sector," Economic Modelling, Elsevier, vol. 96(C), pages 40-53.
    16. Gong, Chengzhu & Gong, Nianjiao & Qi, Rui & Yu, Shiwei, 2020. "Assessment of natural gas supply security in Asia Pacific: Composite indicators with compromise Benefit-of-the-Doubt weights," Resources Policy, Elsevier, vol. 67(C).
    17. Vilarinho, Hermilio & D’Inverno, Giovanna & Nóvoa, Henriqueta & Camanho, Ana S., 2023. "Performance analytics for regulation in retail water utilities: Guiding asset management by identifying peers and targets," Utilities Policy, Elsevier, vol. 82(C).
    18. Fritz Schiltz & Kristof Witte & Deni Mazrekaj, 2020. "Managerial efficiency and efficiency differentials in adult education: a conditional and bias-corrected efficiency analysis," Annals of Operations Research, Springer, vol. 288(2), pages 529-546, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Cordero Ferrera, Jose Manuel & Alonso Morán, Edurne & Nuño Solís, Roberto & Orueta, Juan F. & Souto Arce, Regina, 2013. "Efficiency assessment of primary care providers: A conditional nonparametric approach," MPRA Paper 51926, University Library of Munich, Germany.
    2. Polemis, Michael L. & Tzeremes, Nickolaos G., 2019. "Competitive conditions and sectors’ productive efficiency: A conditional non-parametric frontier analysis," European Journal of Operational Research, Elsevier, vol. 276(3), pages 1104-1118.
    3. Cordero, José Manuel & Alonso-Morán, Edurne & Nuño-Solinis, Roberto & Orueta, Juan F. & Arce, Regina Sauto, 2015. "Efficiency assessment of primary care providers: A conditional nonparametric approach," European Journal of Operational Research, Elsevier, vol. 240(1), pages 235-244.
    4. Bădin, Luiza & Daraio, Cinzia & Simar, Léopold, 2019. "A bootstrap approach for bandwidth selection in estimating conditional efficiency measures," European Journal of Operational Research, Elsevier, vol. 277(2), pages 784-797.
    5. Cordero, Jose M. & Polo, Cristina & Santín, Daniel & Simancas, Rosa, 2018. "Efficiency measurement and cross-country differences among schools: A robust conditional nonparametric analysis," Economic Modelling, Elsevier, vol. 74(C), pages 45-60.
    6. López-Torres, Laura & Johnes, Jill & Elliott, Caroline & Polo, Cristina, 2021. "The effects of competition and collaboration on efficiency in the UK independent school sector," Economic Modelling, Elsevier, vol. 96(C), pages 40-53.
    7. Cordero, José Manuel & Pedraja-Chaparro, Francisco & Pisaflores, Elsa C. & Polo, Cristina, 2016. "Efficiency assessment of Portuguese municipalities using a conditional nonparametric approach," MPRA Paper 70674, University Library of Munich, Germany.
    8. Fritz Schiltz & Kristof Witte & Deni Mazrekaj, 2020. "Managerial efficiency and efficiency differentials in adult education: a conditional and bias-corrected efficiency analysis," Annals of Operations Research, Springer, vol. 288(2), pages 529-546, May.
    9. Cordero, José Manuel & Salinas-Jiménez, Javier & Salinas-Jiménez, M Mar, 2017. "Exploring factors affecting the level of happiness across countries: A conditional robust nonparametric frontier analysis," European Journal of Operational Research, Elsevier, vol. 256(2), pages 663-672.
    10. Fusco, Elisa & Vidoli, Francesco & Sahoo, Biresh K., 2018. "Spatial heterogeneity in composite indicator: A methodological proposal," Omega, Elsevier, vol. 77(C), pages 1-14.
    11. Broadstock, David C. & Matousek, Roman & Meyer, Martin & Tzeremes, Nickolaos G., 2020. "Does corporate social responsibility impact firms' innovation capacity? The indirect link between environmental & social governance implementation and innovation performance," Journal of Business Research, Elsevier, vol. 119(C), pages 99-110.
    12. Jose M. Cordero & Cristina Polo & Daniel Santín, 2020. "Assessment of new methods for incorporating contextual variables into efficiency measures: a Monte Carlo simulation," Operational Research, Springer, vol. 20(4), pages 2245-2265, December.
    13. Luiza Bădin & Cinzia Daraio & Léopold Simar, 2014. "Explaining inefficiency in nonparametric production models: the state of the art," Annals of Operations Research, Springer, vol. 214(1), pages 5-30, March.
    14. José Manuel Cordero & Cristina Polo & Daniel Santín & Gabriela Sicilia, 2016. "Monte-Carlo Comparison of Conditional Nonparametric Methods and Traditional Approaches to Include Exogenous Variables," Pacific Economic Review, Wiley Blackwell, vol. 21(4), pages 483-497, October.
    15. Valentin Zelenyuk, 2019. "Data Envelopment Analysis and Business Analytics: The Big Data Challenges and Some Solutions," CEPA Working Papers Series WP072019, School of Economics, University of Queensland, Australia.
    16. Baležentis, Tomas & De Witte, Kristof, 2015. "One- and multi-directional conditional efficiency measurement – Efficiency in Lithuanian family farms," European Journal of Operational Research, Elsevier, vol. 245(2), pages 612-622.
    17. Calogero Guccio & Domenico Lisi & Marco Martorana & Anna Mignosa, 2017. "On the role of cultural participation in tourism destination performance: an assessment using robust conditional efficiency approach," Journal of Cultural Economics, Springer;The Association for Cultural Economics International, vol. 41(2), pages 129-154, May.
    18. Belmonte-Martin, Irene & Ortiz, Lidia & Polo, Cristina, 2021. "Local tax management in Spain: A study of the conditional efficiency of provincial tax agencies," Socio-Economic Planning Sciences, Elsevier, vol. 78(C).
    19. Amir Moradi-Motlagh & Ali Emrouznejad, 2022. "The origins and development of statistical approaches in non-parametric frontier models: a survey of the first two decades of scholarly literature (1998–2020)," Annals of Operations Research, Springer, vol. 318(1), pages 713-741, November.
    20. D’Inverno, Giovanna & De Witte, Kristof, 2020. "Service level provision in municipalities: A flexible directional distance composite indicator," European Journal of Operational Research, Elsevier, vol. 286(3), pages 1129-1141.

    More about this item

    Keywords

    Education; School district; Organizational effectiveness; Economics of education; Empirical management;
    All these keywords.

    JEL classification:

    • L2 - Industrial Organization - - Firm Objectives, Organization, and Behavior
    • I2 - Health, Education, and Welfare - - Education
    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:267:y:2018:i:3:p:1172-1181. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.