IDEAS home Printed from https://ideas.repec.org/a/spr/annopr/v315y2022i1d10.1007_s10479-022-04629-z.html
   My bibliography  Save this article

Performance evaluation of problematic samples: a robust nonparametric approach for wastewater treatment plants

Author

Listed:
  • Alda A. Henriques

    (Faculdade de Engenharia, Universidade do Porto)

  • Milton Fontes

    (Águas do Centro Litoral (AdP Group))

  • Ana S. Camanho

    (Faculdade de Engenharia, Universidade do Porto)

  • Giovanna D’Inverno

    (Faculty of Economics and Business, KU Leuven)

  • Pedro Amorim

    (Faculdade de Engenharia, Universidade do Porto)

  • Jaime Gabriel Silva

    (AdP Energias and Instituto Superior de Engenharia do Porto)

Abstract

This paper explores robust unconditional and conditional nonparametric approaches to support performance evaluation in problematic samples. Real-world assessments often face critical problems regarding available data, as samples may be relatively small, with high variability in the magnitude of the observed indicators and contextual conditions. This paper explores the possibility of mitigating the impact of potential outlier observations and variability in small samples using a robust nonparametric approach. This approach has the advantage of avoiding unnecessary loss of relevant information, retaining all the decision-making units of the original sample. We devote particular attention to identifying peers and targets in the robust nonparametric approach to guide improvements for underperforming units. The results are compared with a traditional deterministic approach to highlight the proposed method's benefits for problematic samples. This framework's applicability in internal benchmarking studies is illustrated with a case study within the wastewater treatment industry in Portugal.

Suggested Citation

  • Alda A. Henriques & Milton Fontes & Ana S. Camanho & Giovanna D’Inverno & Pedro Amorim & Jaime Gabriel Silva, 2022. "Performance evaluation of problematic samples: a robust nonparametric approach for wastewater treatment plants," Annals of Operations Research, Springer, vol. 315(1), pages 193-220, August.
  • Handle: RePEc:spr:annopr:v:315:y:2022:i:1:d:10.1007_s10479-022-04629-z
    DOI: 10.1007/s10479-022-04629-z
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10479-022-04629-z
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s10479-022-04629-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Rogge, Nicky & De Jaeger, Simon, 2013. "Measuring and explaining the cost efficiency of municipal solid waste collection and processing services," Omega, Elsevier, vol. 41(4), pages 653-664.
    2. Molinos-Senante, María & Hernández-Sancho, Francesc & Mocholí-Arce, Manuel & Sala-Garrido, Ramón, 2014. "Economic and environmental performance of wastewater treatment plants: Potential reductions in greenhouse gases emissions," Resource and Energy Economics, Elsevier, vol. 38(C), pages 125-140.
    3. Léopold Simar & Paul W. Wilson, 1998. "Sensitivity Analysis of Efficiency Scores: How to Bootstrap in Nonparametric Frontier Models," Management Science, INFORMS, vol. 44(1), pages 49-61, January.
    4. Avkiran, Necmi K. & Rowlands, Terry, 2008. "How to better identify the true managerial performance: State of the art using DEA," Omega, Elsevier, vol. 36(2), pages 317-324, April.
    5. Aragon, Y. & Daouia, A. & Thomas-Agnan, C., 2005. "Nonparametric Frontier Estimation: A Conditional Quantile-Based Approach," Econometric Theory, Cambridge University Press, vol. 21(2), pages 358-389, April.
    6. Wilson, Paul W, 1993. "Detecting Outliers in Deterministic Nonparametric Frontier Models with Multiple Outputs," Journal of Business & Economic Statistics, American Statistical Association, vol. 11(3), pages 319-323, July.
    7. Barros, Carlos Pestana, 2008. "Efficiency analysis of hydroelectric generating plants: A case study for Portugal," Energy Economics, Elsevier, vol. 30(1), pages 59-75, January.
    8. Per Andersen & Niels Christian Petersen, 1993. "A Procedure for Ranking Efficient Units in Data Envelopment Analysis," Management Science, INFORMS, vol. 39(10), pages 1261-1264, October.
    9. Sala-Garrido, Ramón & Molinos-Senante, María & Hernández-Sancho, Francesc, 2012. "How does seasonality affect water reuse possibilities? An efficiency and cost analysis," Resources, Conservation & Recycling, Elsevier, vol. 58(C), pages 125-131.
    10. Kristof De Witte & Mika Kortelainen, 2013. "What explains the performance of students in a heterogeneous environment? Conditional efficiency estimation with continuous and discrete environmental variables," Applied Economics, Taylor & Francis Journals, vol. 45(17), pages 2401-2412, June.
    11. C. Vaz & A. Camanho & R. Guimarães, 2010. "The assessment of retailing efficiency using Network Data Envelopment Analysis," Annals of Operations Research, Springer, vol. 173(1), pages 5-24, January.
    12. Fritz Schiltz & Kristof Witte & Deni Mazrekaj, 2020. "Managerial efficiency and efficiency differentials in adult education: a conditional and bias-corrected efficiency analysis," Annals of Operations Research, Springer, vol. 288(2), pages 529-546, May.
    13. Kristof De Witte & Rui Cunha Marques, 2010. "Incorporating heterogeneity in non-parametric models: a methodological comparison," International Journal of Operational Research, Inderscience Enterprises Ltd, vol. 9(2), pages 188-204.
    14. Boaz Golany & Sten Thore, 1997. "Restricted best practice selection in DEA: An overview with a case study evaluating the socio-economic performance of nations," Annals of Operations Research, Springer, vol. 73(0), pages 117-140, October.
    15. Lee, Chia-Yen & Cai, Jia-Ying, 2020. "LASSO variable selection in data envelopment analysis with small datasets," Omega, Elsevier, vol. 91(C).
    16. Fusco, Elisa & Vidoli, Francesco & Rogge, Nicky, 2020. "Spatial directional robust Benefit of the Doubt approach in presence of undesirable output: An application to Italian waste sector," Omega, Elsevier, vol. 94(C).
    17. D’Inverno, Giovanna & Carosi, Laura & Romano, Giulia & Guerrini, Andrea, 2018. "Water pollution in wastewater treatment plants: An efficiency analysis with undesirable output," European Journal of Operational Research, Elsevier, vol. 269(1), pages 24-34.
    18. Sadjadi, S.J. & Omrani, H., 2008. "Data envelopment analysis with uncertain data: An application for Iranian electricity distribution companies," Energy Policy, Elsevier, vol. 36(11), pages 4247-4254, November.
    19. David C. Wheelock & Paul W. Wilson, 2003. "Robust nonparametric estimation of efficiency and technical change in U.S. commercial banking," Working Papers 2003-037, Federal Reserve Bank of St. Louis.
    20. Cinzia Daraio & Léopold Simar, 2005. "Introducing Environmental Variables in Nonparametric Frontier Models: a Probabilistic Approach," Journal of Productivity Analysis, Springer, vol. 24(1), pages 93-121, September.
    21. Léopold Simar, 2003. "Detecting Outliers in Frontier Models: A Simple Approach," Journal of Productivity Analysis, Springer, vol. 20(3), pages 391-424, November.
    22. Carlos Pestana Barros, 2006. "Productivity Growth in the Lisbon Police Force," Public Organization Review, Springer, vol. 6(1), pages 21-35, March.
    23. A S Camanho & R G Dyson, 1999. "Efficiency, size, benchmarks and targets for bank branches: an application of data envelopment analysis," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 50(9), pages 903-915, September.
    24. Kristof Witte & Rui Marques, 2010. "Influential observations in frontier models, a robust non-oriented approach to the water sector," Annals of Operations Research, Springer, vol. 181(1), pages 377-392, December.
    25. Cinzia Daraio & Léopold Simar, 2007. "Conditional nonparametric frontier models for convex and nonconvex technologies: a unifying approach," Journal of Productivity Analysis, Springer, vol. 28(1), pages 13-32, October.
    26. Dyson, R. G. & Allen, R. & Camanho, A. S. & Podinovski, V. V. & Sarrico, C. S. & Shale, E. A., 2001. "Pitfalls and protocols in DEA," European Journal of Operational Research, Elsevier, vol. 132(2), pages 245-259, July.
    27. Charnes, A. & Cooper, W. W. & Rhodes, E., 1978. "Measuring the efficiency of decision making units," European Journal of Operational Research, Elsevier, vol. 2(6), pages 429-444, November.
    28. Daouia, Abdelaati & Simar, Leopold, 2007. "Nonparametric efficiency analysis: A multivariate conditional quantile approach," Journal of Econometrics, Elsevier, vol. 140(2), pages 375-400, October.
    29. Cazals, Catherine & Florens, Jean-Pierre & Simar, Leopold, 2002. "Nonparametric frontier estimation: a robust approach," Journal of Econometrics, Elsevier, vol. 106(1), pages 1-25, January.
    30. De Witte, Kristof & Schiltz, Fritz, 2018. "Measuring and explaining organizational effectiveness of school districts: Evidence from a robust and conditional Benefit-of-the-Doubt approach," European Journal of Operational Research, Elsevier, vol. 267(3), pages 1172-1181.
    31. Ruiz, José L. & Sirvent, Inmaculada, 2019. "Performance evaluation through DEA benchmarking adjusted to goals," Omega, Elsevier, vol. 87(C), pages 150-157.
    32. Guerrini, Andrea & Romano, Giulia & Mancuso, Fabrizio & Carosi, Laura, 2016. "Identifying the performance drivers of wastewater treatment plants through conditional order-m efficiency analysis," Utilities Policy, Elsevier, vol. 42(C), pages 20-31.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Vilarinho, Hermilio & D’Inverno, Giovanna & Nóvoa, Henriqueta & Camanho, Ana S., 2023. "The measurement of asset management performance of water companies," Socio-Economic Planning Sciences, Elsevier, vol. 87(PA).
    2. Pereira, Miguel Alves & Vilarinho, Hermilio & D’Inverno, Giovanna & Camanho, Ana S., 2023. "A regulatory robust conditional approach to measuring the efficiency of wholesale water supply and wastewater treatment services," Utilities Policy, Elsevier, vol. 83(C).
    3. de la Higuera-Molina, Emilio José & Campos-Alba, Cristina María & López-Pérez, Germán & Zafra-Gómez, José Luis, 2023. "Efficiency of water service management alternatives in Spain considering environmental factors," Utilities Policy, Elsevier, vol. 84(C).
    4. Vilarinho, Hermilio & D’Inverno, Giovanna & Nóvoa, Henriqueta & Camanho, Ana S., 2023. "Performance analytics for regulation in retail water utilities: Guiding asset management by identifying peers and targets," Utilities Policy, Elsevier, vol. 82(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Pereira, Miguel Alves & Vilarinho, Hermilio & D’Inverno, Giovanna & Camanho, Ana S., 2023. "A regulatory robust conditional approach to measuring the efficiency of wholesale water supply and wastewater treatment services," Utilities Policy, Elsevier, vol. 83(C).
    2. Henriques, Alda A. & Camanho, Ana S. & Amorim, Pedro & Silva, Jaime G., 2020. "Performance benchmarking using composite indicators to support regulation of the Portuguese wastewater sector," Utilities Policy, Elsevier, vol. 66(C).
    3. Léopold Simar & Paul W. Wilson, 2015. "Statistical Approaches for Non-parametric Frontier Models: A Guided Tour," International Statistical Review, International Statistical Institute, vol. 83(1), pages 77-110, April.
    4. Amir Moradi-Motlagh & Ali Emrouznejad, 2022. "The origins and development of statistical approaches in non-parametric frontier models: a survey of the first two decades of scholarly literature (1998–2020)," Annals of Operations Research, Springer, vol. 318(1), pages 713-741, November.
    5. Wheelock, David C. & Wilson, Paul W., 2008. "Non-parametric, unconditional quantile estimation for efficiency analysis with an application to Federal Reserve check processing operations," Journal of Econometrics, Elsevier, vol. 145(1-2), pages 209-225, July.
    6. Amparo Soler Domínguez & Juan Carlos Matallín Sáez & Emili Tortosa Ausina, 2011. "On the informativeness of persistence for mutual funds' performance evaluation using partial frontiers," Working Papers. Serie EC 2011-08, Instituto Valenciano de Investigaciones Económicas, S.A. (Ivie).
    7. Kristof Witte & Rui Marques, 2010. "Influential observations in frontier models, a robust non-oriented approach to the water sector," Annals of Operations Research, Springer, vol. 181(1), pages 377-392, December.
    8. Fritz Schiltz & Kristof Witte & Deni Mazrekaj, 2020. "Managerial efficiency and efficiency differentials in adult education: a conditional and bias-corrected efficiency analysis," Annals of Operations Research, Springer, vol. 288(2), pages 529-546, May.
    9. Carlucci, Fabio & Corcione, Carlo & Mazzocchi, Paolo & Trincone, Barbara, 2021. "The role of logistics in promoting Italian agribusiness: The Belt and Road Initiative case study," Land Use Policy, Elsevier, vol. 108(C).
    10. Cordero, José Manuel & Alonso-Morán, Edurne & Nuño-Solinis, Roberto & Orueta, Juan F. & Arce, Regina Sauto, 2015. "Efficiency assessment of primary care providers: A conditional nonparametric approach," European Journal of Operational Research, Elsevier, vol. 240(1), pages 235-244.
    11. Bernardino Benito & José Solana & María-Rocío Moreno, 2014. "Explaining efficiency in municipal services providers," Journal of Productivity Analysis, Springer, vol. 42(3), pages 225-239, December.
    12. Caitlin O’Loughlin & Léopold Simar & Paul W. Wilson, 2023. "Methodologies for assessing government efficiency," Chapters, in: António Afonso & João Tovar Jalles & Ana Venâncio (ed.), Handbook on Public Sector Efficiency, chapter 4, pages 72-101, Edward Elgar Publishing.
    13. Nolwenn Roudaut & Anne Vanhems, 2012. "Explaining firms efficiency in the Ivorian manufacturing sector: a robust nonparametric approach," Journal of Productivity Analysis, Springer, vol. 37(2), pages 155-169, April.
    14. Kristof De Witte & Rui Marques, 2010. "Designing performance incentives, an international benchmark study in the water sector," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 18(2), pages 189-220, June.
    15. Cinzia Daraio & Léopold Simar & Paul W. Wilson, 2020. "Fast and efficient computation of directional distance estimators," Annals of Operations Research, Springer, vol. 288(2), pages 805-835, May.
    16. Greco, Salvatore & Ishizaka, Alessio & Tasiou, Menelaos & Torrisi, Gianpiero, 2019. "Sigma-Mu efficiency analysis: A methodology for evaluating units through composite indicators," European Journal of Operational Research, Elsevier, vol. 278(3), pages 942-960.
    17. Cordero Ferrera, Jose Manuel & Alonso Morán, Edurne & Nuño Solís, Roberto & Orueta, Juan F. & Souto Arce, Regina, 2013. "Efficiency assessment of primary care providers: A conditional nonparametric approach," MPRA Paper 51926, University Library of Munich, Germany.
    18. Isabel Narbón-Perpiñá & Kristof De Witte, 2016. "Local governments’ efficiency: A systematic literature review – Part I," Working Papers 2016/20, Economics Department, Universitat Jaume I, Castellón (Spain).
    19. Vilarinho, Hermilio & D’Inverno, Giovanna & Nóvoa, Henriqueta & Camanho, Ana S., 2023. "The measurement of asset management performance of water companies," Socio-Economic Planning Sciences, Elsevier, vol. 87(PA).
    20. Rita Bastião & Nuno de Sousa Pereira, 2020. "Performance in the Delivery of Primary Health Care Services: A Longitudinal Analysis," CEF.UP Working Papers 2002, Universidade do Porto, Faculdade de Economia do Porto.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:annopr:v:315:y:2022:i:1:d:10.1007_s10479-022-04629-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.