IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v274y2019i3p1047-1054.html

Data envelopment analysis and big data

Author

Listed:
  • Khezrimotlagh, Dariush
  • Zhu, Joe
  • Cook, Wade D.
  • Toloo, Mehdi

Abstract

In the traditional data envelopment analysis (DEA) approach for a set of n Decision Making Units (DMUs), a standard DEA model is solved n times, one for each DMU. As the number of DMUs increases, the running-time to solve the standard model sharply rises. In this study, a new framework is proposed to significantly decrease the required DEA calculation time in comparison with the existing methodologies when a large set of DMUs (e.g., 20,000 DMUs or more) is present. The framework includes five steps: (i) selecting a subsample of DMUs using a proposed algorithm, (ii) finding the best-practice DMUs in the selected subsample, (iii) finding the exterior DMUs to the hull of the selected subsample, (iv) identifying the set of all efficient DMUs, and (v) measuring the performance scores of DMUs as those arising from the traditional DEA approach. The variable returns to scale technology is assumed and several simulation experiments are designed to estimate the running-time for applying the proposed method for big data. The obtained results in this study point out that the running-time is decreased up to 99.9% in comparison with the existing techniques. In addition, we illustrate the essential computation time for applying the proposed method as a function of the number of DMUs (cardinality), number of inputs and outputs (dimension), and the proportion of efficient DMUs (density). The methods are also compared on a real data set consisting of 30,099 electric power plants in the United States from 1996 to 2016.

Suggested Citation

  • Khezrimotlagh, Dariush & Zhu, Joe & Cook, Wade D. & Toloo, Mehdi, 2019. "Data envelopment analysis and big data," European Journal of Operational Research, Elsevier, vol. 274(3), pages 1047-1054.
    • Handle: RePEc:eee:ejores:v:274:y:2019:i:3:p:1047-1054
    DOI: 10.1016/j.ejor.2018.10.044
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221718309123
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ejor.2018.10.044?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. J. H. Dulá & R. V. Helgason & N. Venugopal, 1998. "An Algorithm for Identifying the Frame of a Pointed Finite Conical Hull," INFORMS Journal on Computing, INFORMS, vol. 10(3), pages 323-330, August.
    2. R. D. Banker & A. Charnes & W. W. Cooper, 1984. "Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis," Management Science, INFORMS, vol. 30(9), pages 1078-1092, September.
    3. Ali, Agha Iqbal, 1993. "Streamlined computation for data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 64(1), pages 61-67, January.
    4. Richard Barr & Matthew Durchholz, 1997. "Parallel and hierarchical decomposition approaches for solving large-scale Data Envelopment Analysis models," Annals of Operations Research, Springer, vol. 73(0), pages 339-372, October.
    5. Seiford, Lawrence M. & Zhu, Joe, 2005. "A response to comments on modeling undesirable factors in efficiency evaluation," European Journal of Operational Research, Elsevier, vol. 161(2), pages 579-581, March.
    6. Wen-Chih Chen & Sheng-Yung Lai, 2017. "Determining radial efficiency with a large data set by solving small-size linear programs," Annals of Operations Research, Springer, vol. 250(1), pages 147-166, March.
    7. Dula, J. H. & Helgason, R. V., 1996. "A new procedure for identifying the frame of the convex hull of a finite collection of points in multidimensional space," European Journal of Operational Research, Elsevier, vol. 92(2), pages 352-367, July.
    8. William W. Cooper & Lawrence M. Seiford & Kaoru Tone, 2007. "Data Envelopment Analysis," Springer Books, Springer, edition 0, number 978-0-387-45283-8, March.
    9. J.H. Dulá & R.M. Thrall, 2001. "A Computational Framework for Accelerating DEA," Journal of Productivity Analysis, Springer, vol. 16(1), pages 63-78, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Shengqing Chang & Jingjing Ding & Chenpeng Feng & Ruifeng Wang, 2024. "A Hybrid Parallel Processing Strategy for Large-Scale DEA Computation," Computational Economics, Springer;Society for Computational Economics, vol. 63(6), pages 2325-2349, June.
    2. Tao Jie, 2020. "Parallel processing of the Build Hull algorithm to address the large-scale DEA problem," Annals of Operations Research, Springer, vol. 295(1), pages 453-481, December.
    3. Wen-Chih Chen & Sheng-Yung Lai, 2017. "Determining radial efficiency with a large data set by solving small-size linear programs," Annals of Operations Research, Springer, vol. 250(1), pages 147-166, March.
    4. J. H. Dulá, 2011. "An Algorithm for Data Envelopment Analysis," INFORMS Journal on Computing, INFORMS, vol. 23(2), pages 284-296, May.
    5. Chu, Junfei & Rui, Yuting & Khezrimotlagh, Dariush & Zhu, Joe, 2024. "A general computational framework and a hybrid algorithm for large-scale data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 316(2), pages 639-650.
    6. Valentin Zelenyuk, 2019. "Data Envelopment Analysis and Business Analytics: The Big Data Challenges and Some Solutions," CEPA Working Papers Series WP072019, School of Economics, University of Queensland, Australia.
    7. Alexander P. Afanasiev & Vladimir E. Krivonozhko & Andrey V. Lychev & Oleg V. Sukhoroslov, 2020. "Multidimensional frontier visualization based on optimization methods using parallel computations," Journal of Global Optimization, Springer, vol. 76(3), pages 563-574, March.
    8. Xu, Shuqi & Zhu, Qingyuan & Shen, Zhiyang & Vardanyan, Michael & Pan, Yinghao, 2025. "Detecting two-dimensional projection-efficient units in data envelopment analysis under big data scenarios," European Journal of Operational Research, Elsevier, vol. 327(3), pages 957-970.
    9. López, Francisco J., 2011. "Generalizing cross redundancy in data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 214(3), pages 716-721, November.
    10. Wang, Mengyuan & Chen, Ya & Zelenyuk, Valentin, 2025. "DEA for big wide data based on regularization approaches: An application to energy efficiency analysis," Energy Economics, Elsevier, vol. 152(C).
    11. Zelenyuk, Valentin, 2020. "Aggregation of inputs and outputs prior to Data Envelopment Analysis under big data," European Journal of Operational Research, Elsevier, vol. 282(1), pages 172-187.
    12. Peter Fernandes Wanke & Rebecca de Mattos, 2014. "Capacity Issues and Efficiency Drivers in Brazilian Bulk Terminals," Brazilian Business Review, Fucape Business School, vol. 11(5), pages 72-98, October.
    13. Yuanxiang Zhou & Shan Wang & Shuqi Xu & Qingyuan Zhu, 2025. "Big data in data envelopment analysis with undesirable outputs based on simulation and environmental-health matching data of Chinese industrial enterprises," Annals of Operations Research, Springer, vol. 348(1), pages 279-298, May.
    14. Chiu, Yung-Ho & Lee, Jen-Hui & Lu, Ching-Cheng & Shyu, Ming-Kuang & Luo, Zhengying, 2012. "The technology gap and efficiency measure in WEC countries: Application of the hybrid meta frontier model," Energy Policy, Elsevier, vol. 51(C), pages 349-357.
    15. Khezrimotlagh, Dariush & Cook, Wade D. & Zhu, Joe, 2020. "A nonparametric framework to detect outliers in estimating production frontiers," European Journal of Operational Research, Elsevier, vol. 286(1), pages 375-388.
    16. Chien-Ming Chen, 2014. "Evaluating eco-efficiency with data envelopment analysis: an analytical reexamination," Annals of Operations Research, Springer, vol. 214(1), pages 49-71, March.
    17. Dulá, J.H. & López, F.J., 2013. "DEA with streaming data," Omega, Elsevier, vol. 41(1), pages 41-47.
    18. Goodness C. Aye & Giray Gozgor & Rangan Gupta, 2020. "Dynamic and Asymmetric Response of Inequality to Income Volatility: The Case of the United Kingdom," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 147(3), pages 747-762, February.
    19. J.H. Dulá & R.M. Thrall, 2001. "A Computational Framework for Accelerating DEA," Journal of Productivity Analysis, Springer, vol. 16(1), pages 63-78, July.
    20. Franz R. Hahn, 2007. "Determinants of Bank Efficiency in Europe. Assessing Bank Performance Across Markets," WIFO Studies, WIFO, number 31499.

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:274:y:2019:i:3:p:1047-1054. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.