IDEAS home Printed from https://ideas.repec.org/a/spr/busres/v12y2019i2d10.1007_s40685-018-0077-z.html
   My bibliography  Save this article

Is data envelopment analysis a suitable tool for performance measurement and benchmarking in non-production contexts?

Author

Listed:
  • Victoria Wojcik

    (RWTH Aachen University)

  • Harald Dyckhoff

    (RWTH Aachen University)

  • Marcel Clermont

    (TU Braunschweig)

Abstract

After 40 years of research with thousands of application-oriented scientific papers, empirical evidence that data envelopment analysis (DEA) has really improved the practice of performance measurement and benchmarking in real-life non-production contexts is rare. The main reason for this deficit may be that DEA is founded on the concepts of production theory such as production possibility set or returns to scale. These concepts can hardly be applied to pure multiple-criteria evaluation problems, which are often attempted to be solved using DEA. This paper systematically investigates strengths and weaknesses of DEA in the exemplary case of welfare evaluation using real data on 27 countries of the European Union. We analyze and explain the differences in the results of various frequently used DEA models for two different, but strongly connected sets of welfare indicators, thereby demonstrating the pitfalls, which often arise in the application of DEA, as well as some approaches for avoiding them.

Suggested Citation

  • Victoria Wojcik & Harald Dyckhoff & Marcel Clermont, 2019. "Is data envelopment analysis a suitable tool for performance measurement and benchmarking in non-production contexts?," Business Research, Springer;German Academic Association for Business Research, vol. 12(2), pages 559-595, December.
    • Handle: RePEc:spr:busres:v:12:y:2019:i:2:d:10.1007_s40685-018-0077-z
    DOI: 10.1007/s40685-018-0077-z
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s40685-018-0077-z
    File Function: Abstract
    Download Restriction: no
    File URL: https://libkey.io/10.1007/s40685-018-0077-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Bohringer, Christoph & Jochem, Patrick E.P., 2007. "Measuring the immeasurable -- A survey of sustainability indices," Ecological Economics, Elsevier, vol. 63(1), pages 1-8, June.
    2. Per Andersen & Niels Christian Petersen, 1993. "A Procedure for Ranking Efficient Units in Data Envelopment Analysis," Management Science, INFORMS, vol. 39(10), pages 1261-1264, October.
    3. Charnes, A. & Cooper, W. W. & Rhodes, E., 1978. "Measuring the efficiency of decision making units," European Journal of Operational Research, Elsevier, vol. 2(6), pages 429-444, November.
    4. Victoria Wojcik & Harald Dyckhoff & Sebastian Gutgesell, 2017. "The desirable input of undesirable factors in data envelopment analysis," Annals of Operations Research, Springer, vol. 259(1), pages 461-484, December.
    5. Dyson, R. G. & Allen, R. & Camanho, A. S. & Podinovski, V. V. & Sarrico, C. S. & Shale, E. A., 2001. "Pitfalls and protocols in DEA," European Journal of Operational Research, Elsevier, vol. 132(2), pages 245-259, July.
    6. Emrouznejad, Ali & De Witte, Kristof, 2010. "COOPER-framework: A unified process for non-parametric projects," European Journal of Operational Research, Elsevier, vol. 207(3), pages 1573-1586, December.
    7. Emrouznejad, A. & De Witte, K., 2010. "COOPER-framework: A Unified Standard Process for Non-parametric Projects," Working Papers 18, Top Institute for Evidence Based Education Research.
    8. Harald Dyckhoff, 2018. "Multi-criteria production theory: foundation of non-financial and sustainability performance evaluation," Journal of Business Economics, Springer, vol. 88(7), pages 851-882, September.
    9. D K Despotis, 2005. "A reassessment of the human development index via data envelopment analysis," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 56(8), pages 969-980, August.
    10. Antonio Jurado & Jesus Perez-Mayo, 2012. "Construction and Evolution of a Multidimensional Well-Being Index for the Spanish Regions," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 107(2), pages 259-279, June.
    11. Tone, Kaoru, 2002. "A slacks-based measure of super-efficiency in data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 143(1), pages 32-41, November.
    12. Timothy J. Coelli & D.S. Prasada Rao & Christopher J. O’Donnell & George E. Battese, 2005. "An Introduction to Efficiency and Productivity Analysis," Springer Books, Springer, edition 0, number 978-0-387-25895-9, June.
    13. Mónica Domínguez-Serrano & Francisco Blancas, 2011. "A Gender Wellbeing Composite Indicator: The Best-Worst Global Evaluation Approach," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 102(3), pages 477-496, July.
    14. R. D. Banker & A. Charnes & W. W. Cooper, 1984. "Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis," Management Science, INFORMS, vol. 30(9), pages 1078-1092, September.
    15. Ramanathan, Ramakrishnan, 2006. "Evaluating the comparative performance of countries of the Middle East and North Africa: A DEA application," Socio-Economic Planning Sciences, Elsevier, vol. 40(2), pages 156-167, June.
    16. Tone, Kaoru, 2001. "A slacks-based measure of efficiency in data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 130(3), pages 498-509, May.
    17. B. Hollingsworth & P. Smith, 2003. "Use of ratios in data envelopment analysis," Applied Economics Letters, Taylor & Francis Journals, vol. 10(11), pages 733-735.
    18. Stéphane Blancard & Jean-François Hoarau, 2011. "Optimizing the new formulation of the United Nations' human development index: An empirical view from data envelopment analysis," Economics Bulletin, AccessEcon, vol. 31(1), pages 989-1003.
    19. Bougnol, M.-L. & Dulá, J.H. & Estellita Lins, M.P. & Moreira da Silva, A.C., 2010. "Enhancing standard performance practices with DEA," Omega, Elsevier, vol. 38(1-2), pages 33-45, February.
    20. Joe Zhu, 2014. "Data Envelopment Analysis," International Series in Operations Research & Management Science, in: Quantitative Models for Performance Evaluation and Benchmarking, edition 3, chapter 1, pages 1-9, Springer.
    21. Pilar Murias & Fidel Martinez & Carlos Miguel, 2006. "An Economic Wellbeing Index for the Spanish Provinces: A Data Envelopment Analysis Approach," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 77(3), pages 395-417, July.
    22. Po-Chin Wu & Chiung-Wen Fan & Sheng-Chieh Pan, 2014. "Does Human Development Index Provide Rational Development Rankings? Evidence from Efficiency Rankings in Super Efficiency Model," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 116(2), pages 647-658, April.
    23. Cook, Wade D. & Tone, Kaoru & Zhu, Joe, 2014. "Data envelopment analysis: Prior to choosing a model," Omega, Elsevier, vol. 44(C), pages 1-4.
    24. Olesen, Ole Bent & Petersen, Niels Christian & Podinovski, Victor V., 2015. "Efficiency analysis with ratio measures," European Journal of Operational Research, Elsevier, vol. 245(2), pages 446-462.
    25. Charnes, A. & Cooper, W. W. & Golany, B. & Seiford, L. & Stutz, J., 1985. "Foundations of data envelopment analysis for Pareto-Koopmans efficient empirical production functions," Journal of Econometrics, Elsevier, vol. 30(1-2), pages 91-107.
    26. Thompson, Russell G. & Langemeier, Larry N. & Lee, Chih-Tah & Lee, Euntaik & Thrall, Robert M., 1990. "The role of multiplier bounds in efficiency analysis with application to Kansas farming," Journal of Econometrics, Elsevier, vol. 46(1-2), pages 93-108.
    27. Stéphane Blancard & Jean-François Hoarau, 2011. "Optimizing the new formulation of the United Nations' human development index: An empirical view from data envelopment analysis," Economics Bulletin, AccessEcon, vol. 31(1), pages 989-1003.
    28. Olesen, Ole Bent & Petersen, Niels Christian & Podinovski, Victor V., 2017. "Efficiency measures and computational approaches for data envelopment analysis models with ratio inputs and outputs," European Journal of Operational Research, Elsevier, vol. 261(2), pages 640-655.
    29. Orla A. Murphy & Ping Wang & Sunny X. Wang & Greg Tkacz, 2013. "An economic efficiency study on different regions of Ghana via Slack-based data envelopment analysis and regression analysis," Applied Economics, Taylor & Francis Journals, vol. 45(34), pages 4773-4780, December.
    30. Dyckhoff, H. & Allen, K., 2001. "Measuring ecological efficiency with data envelopment analysis (DEA)," European Journal of Operational Research, Elsevier, vol. 132(2), pages 312-325, July.
    31. Miki Malul & Yossi Hadad & Avner Ben‐Yair, 2009. "Measuring and ranking of economic, environmental and social efficiency of countries," International Journal of Social Economics, Emerald Group Publishing Limited, vol. 36(8), pages 832-843, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Huy Tung Pham & Luong Hai Nguyen, 2022. "Empirical Performance Measurement of Cargo Handling Equipment in Vietnam Container Terminals," Logistics, MDPI, vol. 6(3), pages 1-18, July.
    2. Benedykt OPALKA, 2023. "Determinants of Public Investment Effectiveness Evaluation in Unstable Macroeconomic Environment," RAIS Journal for Social Sciences, Research Association for Interdisciplinary Studies, vol. 7(2), pages 40-49, November.
    3. Rajko Tomaš, 2022. "Measurement of the Concentration of Potential Quality of Life in Local Communities," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 163(1), pages 79-109, August.
    4. Mansour Zarrin & Jan Schoenfelder & Jens O. Brunner, 2022. "Homogeneity and Best Practice Analyses in Hospital Performance Management: An Analytical Framework," Health Care Management Science, Springer, vol. 25(3), pages 406-425, September.
    5. Shoaib Aslam & Mohamed H. Elmagrhi & Ramiz Ur Rehman & Collins G. Ntim, 2021. "Environmental management practices and financial performance using data envelopment analysis in Japan: The mediating role of environmental performance," Business Strategy and the Environment, Wiley Blackwell, vol. 30(4), pages 1655-1673, May.
    6. Lisciandra, Maurizio & Milani, Riccardo & Millemaci, Emanuele, 2022. "A corruption risk indicator for public procurement," European Journal of Political Economy, Elsevier, vol. 73(C).
    7. Dyckhoff, Harald & Souren, Rainer, 2022. "Integrating multiple criteria decision analysis and production theory for performance evaluation: Framework and review," European Journal of Operational Research, Elsevier, vol. 297(3), pages 795-816.
    8. Heidari, Mohammad Davoud & Turner, Ian & Ardestani-Jaafari, Amir & Pelletier, Nathan, 2021. "Operations research for environmental assessment of crop-livestock production systems," Agricultural Systems, Elsevier, vol. 193(C).
    9. Salvatore Loprevite & Domenico Raucci & Daniela Rupo, 2020. "KPIs Reporting and Financial Performance in the Transition to Mandatory Disclosure: The Case of Italy," Sustainability, MDPI, vol. 12(12), pages 1-24, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mariano, Enzo Barberio & Sobreiro, Vinicius Amorim & Rebelatto, Daisy Aparecida do Nascimento, 2015. "Human development and data envelopment analysis: A structured literature review," Omega, Elsevier, vol. 54(C), pages 33-49.
    2. Mai, Nhat Chi, 2015. "Efficiency of the banking system in Vietnam under financial liberalization," OSF Preprints qsf6d, Center for Open Science.
    3. Enzo Barberio Mariano & Diogo Ferraz & Simone Cristina Oliveira Gobbo, 2021. "The Human Development Index with Multiple Data Envelopment Analysis Approaches: A Comparative Evaluation Using Social Network Analysis," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 157(2), pages 443-500, September.
    4. Harald Dyckhoff, 2018. "Multi-criteria production theory: foundation of non-financial and sustainability performance evaluation," Journal of Business Economics, Springer, vol. 88(7), pages 851-882, September.
    5. Dariush Khezrimotlagh & Wade D. Cook & Joe Zhu, 2021. "Number of performance measures versus number of decision making units in DEA," Annals of Operations Research, Springer, vol. 303(1), pages 529-562, August.
    6. Lampe, Hannes W. & Hilgers, Dennis, 2015. "Trajectories of efficiency measurement: A bibliometric analysis of DEA and SFA," European Journal of Operational Research, Elsevier, vol. 240(1), pages 1-21.
    7. Dyckhoff, Harald & Souren, Rainer, 2022. "Integrating multiple criteria decision analysis and production theory for performance evaluation: Framework and review," European Journal of Operational Research, Elsevier, vol. 297(3), pages 795-816.
    8. Javad Gerami & Mohammad Reza Mozaffari & P. F. Wanke & Henrique Correa, 2022. "A novel slacks-based model for efficiency and super-efficiency in DEA-R," Operational Research, Springer, vol. 22(4), pages 3373-3410, September.
    9. Jorge Guardiola & Andrés J. Picazo-Tadeo, 2013. "Weighting life domains with Data Envelopment Analysis," Working Papers 1311, Department of Applied Economics II, Universidad de Valencia.
    10. Vincenzo Patrizii & Anna Pettini & Giuliano Resce, 2017. "The Cost of Well-Being," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 133(3), pages 985-1010, September.
    11. Jorge Guardiola & Andrés Picazo-Tadeo, 2014. "Building Weighted-Domain Composite Indices of Life Satisfaction with Data Envelopment Analysis," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 117(1), pages 257-274, May.
    12. Necmi Kemal Avkiran, 2017. "An illustration of multiple-stakeholder perspective using a survey across Australia, China and Japan," Annals of Operations Research, Springer, vol. 248(1), pages 93-121, January.
    13. Rashidi, Kamran & Cullinane, Kevin, 2019. "Evaluating the sustainability of national logistics performance using Data Envelopment Analysis," Transport Policy, Elsevier, vol. 74(C), pages 35-46.
    14. Jesús Peiró-Palomino & Andrés J. Picazo-Tadeo, 2018. "OECD: One or Many? Ranking Countries with a Composite Well-Being Indicator," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 139(3), pages 847-869, October.
    15. Sahoo, Biresh K. & Singh, Ramadhar & Mishra, Bineet & Sankaran, Krithiga, 2017. "Research productivity in management schools of India during 1968-2015: A directional benefit-of-doubt model analysis," Omega, Elsevier, vol. 66(PA), pages 118-139.
    16. Chen, Chien-Ming, 2013. "Super efficiencies or super inefficiencies? Insights from a joint computation model for slacks-based measures in DEA," European Journal of Operational Research, Elsevier, vol. 226(2), pages 258-267.
    17. Arabi, Behrouz & Munisamy, Susila & Emrouznejad, Ali & Toloo, Mehdi & Ghazizadeh, Mohammad Sadegh, 2016. "Eco-efficiency considering the issue of heterogeneity among power plants," Energy, Elsevier, vol. 111(C), pages 722-735.
    18. Diogo Ferraz & Enzo B. Mariano & Daisy Rebelatto & Dominik Hartmann, 2020. "Linking Human Development and the Financial Responsibility of Regions: Combined Index Proposals Using Methods from Data Envelopment Analysis," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 150(2), pages 439-478, July.
    19. Sebastian Kohl & Jan Schoenfelder & Andreas Fügener & Jens O. Brunner, 2019. "The use of Data Envelopment Analysis (DEA) in healthcare with a focus on hospitals," Health Care Management Science, Springer, vol. 22(2), pages 245-286, June.
    20. Rafael Benítez & Vicente Coll-Serrano & Vicente J. Bolós, 2021. "deaR-Shiny: An Interactive Web App for Data Envelopment Analysis," Sustainability, MDPI, vol. 13(12), pages 1-19, June.

    More about this item

    Keywords

    Benchmarking; Performance measurement; Data envelopment analysis application; Model choice and building; Indicator selection; Welfare evaluation;
    All these keywords.

    JEL classification:

    • C61 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Optimization Techniques; Programming Models; Dynamic Analysis
    • C67 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Input-Output Models
    • I31 - Health, Education, and Welfare - - Welfare, Well-Being, and Poverty - - - General Welfare, Well-Being

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:busres:v:12:y:2019:i:2:d:10.1007_s40685-018-0077-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.