IDEAS home Printed from https://ideas.repec.org/a/spr/schmbr/v71y2019i4d10.1007_s41464-019-00078-7.html
   My bibliography  Save this article

Identification of Outliers in Data Envelopment Analysis

Author

Listed:
  • Marcel Clermont

    (Duale Hochschule Gera-Eisenach)

  • Julia Schaefer

    (RWTH Aachen University)

Abstract

Data Envelopment Analysis (DEA) is a deterministic method for the aggregation of multidimensional measures and subsequent efficiency analysis. Due to its inherent determinism, however, it reacts sensitively to outliers in datasets. Existing methods for identifying such outliers have two main disadvantages. First, from a more conceptional point of view, a uniform definition of an outlier is missing. Second, there are technical disadvantages of each method. For instance, arbitrarily limited values have to be set by the user, like the amount of efficiency value from which on a decision making unit is regarded as an outlier. This paper initially presents a definition of outliers, which explicitly takes the specifics of DEA into account. Based on this definition, an approach for identifying outliers in DEA is introduced which explicitly tackles the technical disadvantages and takes them into account in the developed algorithm. The plausibility of this approach is validated on the basis of empirical examples from performance measurement at the university level.

Suggested Citation

  • Marcel Clermont & Julia Schaefer, 2019. "Identification of Outliers in Data Envelopment Analysis," Schmalenbach Business Review, Springer;Schmalenbach-Gesellschaft, vol. 71(4), pages 475-496, October.
    • Handle: RePEc:spr:schmbr:v:71:y:2019:i:4:d:10.1007_s41464-019-00078-7
    DOI: 10.1007/s41464-019-00078-7
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s41464-019-00078-7
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s41464-019-00078-7?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Zijiang Yang & Xiaogang Wang & Dongming Sun, 2010. "Using the bootstrap method to detect influential DMUs in data envelopment analysis," Annals of Operations Research, Springer, vol. 173(1), pages 89-103, January.
    2. Rafael Ball & Bernhard Mittermaier & Dirk Tunger, 2009. "Creation of journal-based publication profiles of scientific institutions — A methodology for the interdisciplinary comparison of scientific research based on the J-factor," Scientometrics, Springer;Akadémiai Kiadó, vol. 81(2), pages 381-392, November.
    3. Golany, B & Roll, Y, 1989. "An application procedure for DEA," Omega, Elsevier, vol. 17(3), pages 237-250.
    4. Marcel Clermont & Alexander Dirksen & Harald Dyckhoff, 2015. "Returns to scale of Business Administration research in Germany," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(2), pages 583-614, May.
    5. Lampe, Hannes W. & Hilgers, Dennis, 2015. "Trajectories of efficiency measurement: A bibliometric analysis of DEA and SFA," European Journal of Operational Research, Elsevier, vol. 240(1), pages 1-21.
    6. Per Andersen & Niels Christian Petersen, 1993. "A Procedure for Ranking Efficient Units in Data Envelopment Analysis," Management Science, INFORMS, vol. 39(10), pages 1261-1264, October.
    7. Joe Zhu, 2014. "Quantitative Models for Performance Evaluation and Benchmarking," International Series in Operations Research and Management Science, Springer, edition 3, number 978-3-319-06647-9, September.
    8. Marcel Clermont, 2016. "Effectiveness and efficiency of research in Germany over time: an analysis of German business schools between 2001 and 2009," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(3), pages 1347-1381, September.
    9. A. T. Flegg & D. O. Allen, 2007. "Does Expansion Cause Congestion? The Case of the Older British Universities, 1994-2004," Education Economics, Taylor & Francis Journals, vol. 15(1), pages 75-102.
    10. F. Hosseinzadeh Lotfi & G. R. Jahanshahloo & M. Khodabakhshi & M. Rostamy-Malkhlifeh & Z. Moghaddas & M. Vaez-Ghasemi, 2013. "A Review of Ranking Models in Data Envelopment Analysis," Journal of Applied Mathematics, Hindawi, vol. 2013, pages 1-20, July.
    11. Worthington, Andrew C. & Lee, Boon L., 2008. "Efficiency, technology and productivity change in Australian universities, 1998-2003," Economics of Education Review, Elsevier, vol. 27(3), pages 285-298, June.
    12. Dilger, Alexander & Müller, Harry, 2011. "Ein Forschungsleistungsranking auf der Grundlage von Google Scholar," Discussion Papers of the Institute for Organisational Economics 12/2011, University of Münster, Institute for Organisational Economics.
    13. Frey Bruno S., 2007. "Evaluierungen, Evaluierungen . . . Evaluitis," Perspektiven der Wirtschaftspolitik, De Gruyter, vol. 8(3), pages 207-220, August.
    14. Jradi, Samah & Ruggiero, John, 2019. "Stochastic data envelopment analysis: A quantile regression approach to estimate the production frontier," European Journal of Operational Research, Elsevier, vol. 278(2), pages 385-393.
    15. Avkiran, Necmi K., 2001. "Investigating technical and scale efficiencies of Australian Universities through data envelopment analysis," Socio-Economic Planning Sciences, Elsevier, vol. 35(1), pages 57-80, March.
    16. Maria Sousa & Borko Stošić, 2005. "Technical Efficiency of the Brazilian Municipalities: Correcting Nonparametric Frontier Measurements for Outliers," Journal of Productivity Analysis, Springer, vol. 24(2), pages 157-181, October.
    17. Liu, John S. & Lu, Louis Y.Y. & Lu, Wen-Min & Lin, Bruce J.Y., 2013. "A survey of DEA applications," Omega, Elsevier, vol. 41(5), pages 893-902.
    18. Ralph L. Keeney & Kelly E. See & Detlof von Winterfeldt, 2006. "Evaluating Academic Programs: With Applications to U.S. Graduate Decision Science Programs," Operations Research, INFORMS, vol. 54(5), pages 813-828, October.
    19. Tommaso Agasisti & Giuseppe Munda & Ralph Hippe, 2019. "Measuring the efficiency of European education systems by combining Data Envelopment Analysis and Multiple-Criteria Evaluation," Journal of Productivity Analysis, Springer, vol. 51(2), pages 105-124, June.
    20. Charnes, A. & Cooper, W. W. & Rhodes, E., 1978. "Measuring the efficiency of decision making units," European Journal of Operational Research, Elsevier, vol. 2(6), pages 429-444, November.
    21. Glenn Milligan & Martha Cooper, 1985. "An examination of procedures for determining the number of clusters in a data set," Psychometrika, Springer;The Psychometric Society, vol. 50(2), pages 159-179, June.
    22. George Emm Halkos & Nickolaos G. Tzeremes, 2011. "Measuring economic journals’ citation efficiency: a data envelopment analysis approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(3), pages 979-1001, September.
    23. Douglas M. Hawkins, 1980. "Critical Values for Identifying Outliers," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 29(1), pages 95-96, March.
    24. Meeusen, Wim & van den Broeck, Julien, 1977. "Efficiency Estimation from Cobb-Douglas Production Functions with Composed Error," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 18(2), pages 435-444, June.
    25. Maria Olivares & Andrea Schenker-Wicki, 2012. "The Dynamics of Productivity in the Swiss and German University Sector: A Non-Parametric Analysis that Accounts for Heterogeneous Production," Working Papers 309, University of Zurich, Department of Business Administration (IBW).
    26. Yiannis G. Smirlis & Dimitris K. Despotis, 2012. "Relaxing The Impact Of Extreme Units In Data Envelopment Analysis," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 11(05), pages 893-907.
    27. Ali Bahari & Ali Emrouznejad, 2014. "Influential DMUs and outlier detection in data envelopment analysis with an application to health care," Annals of Operations Research, Springer, vol. 223(1), pages 95-108, December.
    28. Joe Zhu, 2014. "Data Envelopment Analysis," International Series in Operations Research & Management Science, in: Quantitative Models for Performance Evaluation and Benchmarking, edition 3, chapter 1, pages 1-9, Springer.
    29. Banker, Rajiv D. & Chang, Hsihui, 2006. "The super-efficiency procedure for outlier identification, not for ranking efficient units," European Journal of Operational Research, Elsevier, vol. 175(2), pages 1311-1320, December.
    30. Léopold Simar & Paul W. Wilson, 1998. "Sensitivity Analysis of Efficiency Scores: How to Bootstrap in Nonparametric Frontier Models," Management Science, INFORMS, vol. 44(1), pages 49-61, January.
    31. Aragon, Y. & Daouia, A. & Thomas-Agnan, C., 2005. "Nonparametric Frontier Estimation: A Conditional Quantile-Based Approach," Econometric Theory, Cambridge University Press, vol. 21(2), pages 358-389, April.
    32. Julia Schaefer & Marcel Clermont, 2018. "Stochastic non-smooth envelopment of data for multi-dimensional output," Journal of Productivity Analysis, Springer, vol. 50(3), pages 139-154, December.
    33. Horne, Jocelyn & Hu, Baiding, 2008. "Estimation of cost efficiency of Australian universities," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 78(2), pages 266-275.
    34. Liu, Hui-hui & Song, Yao-yao & Yang, Guo-liang, 2019. "Cross-efficiency evaluation in data envelopment analysis based on prospect theory," European Journal of Operational Research, Elsevier, vol. 273(1), pages 364-375.
    35. Léopold Simar, 2003. "Detecting Outliers in Frontier Models: A Simple Approach," Journal of Productivity Analysis, Springer, vol. 20(3), pages 391-424, November.
    36. Johnes, Jill, 2006. "Data envelopment analysis and its application to the measurement of efficiency in higher education," Economics of Education Review, Elsevier, vol. 25(3), pages 273-288, June.
    37. Charnes, A. & Cooper, W. W. & Golany, B. & Seiford, L. & Stutz, J., 1985. "Foundations of data envelopment analysis for Pareto-Koopmans efficient empirical production functions," Journal of Econometrics, Elsevier, vol. 30(1-2), pages 91-107.
    38. Kristof Witte & Rui Marques, 2010. "Influential observations in frontier models, a robust non-oriented approach to the water sector," Annals of Operations Research, Springer, vol. 181(1), pages 377-392, December.
    39. R. D. Banker & A. Charnes & W. W. Cooper, 1984. "Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis," Management Science, INFORMS, vol. 30(9), pages 1078-1092, September.
    40. A L Johnson & L F McGinnis, 2009. "The hyperbolic-oriented efficiency measure as a remedy to infeasibility of super efficiency models," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 60(11), pages 1511-1517, November.
    41. E Thanassoulis & M Kortelainen & G Johnes & J Johnes, 2011. "Costs and efficiency of higher education institutions in England: a DEA analysis," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 62(7), pages 1282-1297, July.
    42. Cazals, Catherine & Florens, Jean-Pierre & Simar, Leopold, 2002. "Nonparametric frontier estimation: a robust approach," Journal of Econometrics, Elsevier, vol. 106(1), pages 1-25, January.
    43. J Ondrich & J Ruggiero, 2002. "Outlier detection in data envelopment analysis: an analysis of jackknifing," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 53(3), pages 342-346, March.
    44. Thomas Bolli & Mehdi Farsi, 2011. "The dynamics of labor productivity in Swiss universities," KOF Working papers 11-278, KOF Swiss Economic Institute, ETH Zurich.
    45. Tone, Kaoru, 2001. "A slacks-based measure of efficiency in data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 130(3), pages 498-509, May.
    46. Thomas Bolli & Mehdi Farsi, 2015. "The dynamics of productivity in Swiss universities," Journal of Productivity Analysis, Springer, vol. 44(1), pages 21-38, August.
    47. Zara Daghbashyan & Enrico Deiaco & Maureen McKelvey, 2014. "How and why does cost efficiency of universities differ across European countries? An explorative attempt using new microdata," Chapters, in: Andrea Bonaccorsi (ed.), Knowledge, Diversity and Performance in European Higher Education, chapter 9, pages iii-iii, Edward Elgar Publishing.
    48. Fouchet Robert & Marcel Guenoun, 2007. "Performance management in intermunicipal authorities," Post-Print hal-01802368, HAL.
    49. Aigner, Dennis & Lovell, C. A. Knox & Schmidt, Peter, 1977. "Formulation and estimation of stochastic frontier production function models," Journal of Econometrics, Elsevier, vol. 6(1), pages 21-37, July.
    50. Fandel, Gunter, 2007. "On the performance of universities in North Rhine-Westphalia, Germany: Government's redistribution of funds judged using DEA efficiency measures," European Journal of Operational Research, Elsevier, vol. 176(1), pages 521-533, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Andreas Dellnitz & Madjid Tavana & Rajiv Banker, 2023. "A novel median-based optimization model for eco-efficiency assessment in data envelopment analysis," Annals of Operations Research, Springer, vol. 322(2), pages 661-690, March.
    2. Chen Cao & Xueyun Chen, 2021. "Can Industrial Integration Improve the Sustainability of Grain Security?," Sustainability, MDPI, vol. 13(24), pages 1-17, December.
    3. Tommaso Agasisti & Guo-liang Yang & Yao-yao Song & Carolyn-Thi Thanh Dung Tran, 2021. "Evaluating the higher education productivity of Chinese and European “elite” universities using a meta-frontier approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 5819-5853, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Marcel Clermont, 2016. "Effectiveness and efficiency of research in Germany over time: an analysis of German business schools between 2001 and 2009," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(3), pages 1347-1381, September.
    2. Amir Moradi-Motlagh & Ali Emrouznejad, 2022. "The origins and development of statistical approaches in non-parametric frontier models: a survey of the first two decades of scholarly literature (1998–2020)," Annals of Operations Research, Springer, vol. 318(1), pages 713-741, November.
    3. Kristof De Witte & Laura López-Torres, 2017. "Efficiency in education: a review of literature and a way forward," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 68(4), pages 339-363, April.
    4. Johnes, Jill, 2015. "Operational Research in education," European Journal of Operational Research, Elsevier, vol. 243(3), pages 683-696.
    5. Rita Bastião & Nuno de Sousa Pereira, 2020. "Performance in the Delivery of Primary Health Care Services: A Longitudinal Analysis," CEF.UP Working Papers 2002, Universidade do Porto, Faculdade de Economia do Porto.
    6. Emrouznejad, Ali & De Witte, Kristof, 2010. "COOPER-framework: A unified process for non-parametric projects," European Journal of Operational Research, Elsevier, vol. 207(3), pages 1573-1586, December.
    7. Manuel Salas-Velasco, 2020. "Measuring and explaining the production efficiency of Spanish universities using a non-parametric approach and a bootstrapped-truncated regression," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(2), pages 825-846, February.
    8. Sanjeet Singh & Prabhat Ranjan, 2018. "Efficiency analysis of non-homogeneous parallel sub-unit systems for the performance measurement of higher education," Annals of Operations Research, Springer, vol. 269(1), pages 641-666, October.
    9. Lampe, Hannes W. & Hilgers, Dennis, 2015. "Trajectories of efficiency measurement: A bibliometric analysis of DEA and SFA," European Journal of Operational Research, Elsevier, vol. 240(1), pages 1-21.
    10. Mehdi Rhaiem, 2017. "Measurement and determinants of academic research efficiency: a systematic review of the evidence," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(2), pages 581-615, February.
    11. Kristof Witte & Rui Marques, 2010. "Influential observations in frontier models, a robust non-oriented approach to the water sector," Annals of Operations Research, Springer, vol. 181(1), pages 377-392, December.
    12. Wu, Yunna & Ke, Yiming & Zhang, Ting & Liu, Fangtong & Wang, Jing, 2018. "Performance efficiency assessment of photovoltaic poverty alleviation projects in China: A three-phase data envelopment analysis model," Energy, Elsevier, vol. 159(C), pages 599-610.
    13. Léopold Simar & Paul W. Wilson, 2015. "Statistical Approaches for Non-parametric Frontier Models: A Guided Tour," International Statistical Review, International Statistical Institute, vol. 83(1), pages 77-110, April.
    14. Ruiz, José L. & Segura, José V. & Sirvent, Inmaculada, 2015. "Benchmarking and target setting with expert preferences: An application to the evaluation of educational performance of Spanish universities," European Journal of Operational Research, Elsevier, vol. 242(2), pages 594-605.
    15. Mai, Nhat Chi, 2015. "Efficiency of the banking system in Vietnam under financial liberalization," OSF Preprints qsf6d, Center for Open Science.
    16. Sebastian Kohl & Jan Schoenfelder & Andreas Fügener & Jens O. Brunner, 2019. "The use of Data Envelopment Analysis (DEA) in healthcare with a focus on hospitals," Health Care Management Science, Springer, vol. 22(2), pages 245-286, June.
    17. Calogero Guccio & Marco Ferdinando Martorana & Luisa Monaco, 2016. "Evaluating the impact of the Bologna Process on the efficiency convergence of Italian universities: a non-parametric frontier approach," Journal of Productivity Analysis, Springer, vol. 45(3), pages 275-298, June.
    18. Agrell, Per J. & Niknazar, Pooria, 2014. "Structural and behavioral robustness in applied best-practice regulation," Socio-Economic Planning Sciences, Elsevier, vol. 48(1), pages 89-103.
    19. Heinz Ahn & Marcel Clermont & Julia Langner, 2022. "The impact of selected input and output factors on measuring research efficiency of university research fields: insights from a purpose-, field-, and method-specific perspective," Journal of Business Economics, Springer, vol. 92(8), pages 1303-1335, October.
    20. Sahoo, Biresh K. & Singh, Ramadhar & Mishra, Bineet & Sankaran, Krithiga, 2017. "Research productivity in management schools of India during 1968-2015: A directional benefit-of-doubt model analysis," Omega, Elsevier, vol. 66(PA), pages 118-139.

    More about this item

    Keywords

    Data Envelopment Analysis; Outlier detection; Efficiency analysis; Performance measurement; Cluster analysis;
    All these keywords.

    JEL classification:

    • C67 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Input-Output Models
    • M11 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - Business Administration - - - Production Management
    • M49 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - Accounting - - - Other
    • P47 - Political Economy and Comparative Economic Systems - - Other Economic Systems - - - Performance and Prospects

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:schmbr:v:71:y:2019:i:4:d:10.1007_s41464-019-00078-7. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.