IDEAS home Printed from https://ideas.repec.org/p/qld/uqcepa/137.html
   My bibliography  Save this paper

Data Envelopment Analysis and Business Analytics: The Big Data Challenges and Some Solutions

Author

Abstract

The goal of this article is three-fold. The first goal is to present a concise review of Data Envelopment Analysis (DEA) for the more general Business Analytics (BA) community. The second goal of this paper is to discuss the key aspect (and thus the key challenge) of BA—the ‘big data’—to the DEA community, which besides a few exceptions, appears to have been largely circumventing this area, despite it gaining more and more attention in other areas of research and practice. The third, and most important, goal of this paper is to discuss possible solutions to the ‘big data’ problem related to the large dimensions in the context of DEA. To achieve the latter goal, we presented some theoretical grounds and performed a new simulation study to explore the price-based aggregation as a solution to address one of the key challenges of the ‘big data’ problems for DEA—the immense dimensionality problem.

Suggested Citation

  • Valentin Zelenyuk, 2019. "Data Envelopment Analysis and Business Analytics: The Big Data Challenges and Some Solutions," CEPA Working Papers Series WP072019, School of Economics, University of Queensland, Australia.
  • Handle: RePEc:qld:uqcepa:137
    as

    Download full text from publisher

    File URL: https://economics.uq.edu.au/files/14356/WP072019.pdf
    Download Restriction: no

    References listed on IDEAS

    as
    1. Leopold Simar & Paul Wilson, 2000. "A general methodology for bootstrapping in non-parametric frontier models," Journal of Applied Statistics, Taylor & Francis Journals, vol. 27(6), pages 779-802.
    2. Léopold Simar & Paul Wilson, 2000. "Statistical Inference in Nonparametric Frontier Models: The State of the Art," Journal of Productivity Analysis, Springer, vol. 13(1), pages 49-78, January.
    3. Diewert, W Erwin, 1983. " The Measurement of Waste within the Production Sector of an Open Economy," Scandinavian Journal of Economics, Wiley Blackwell, vol. 85(2), pages 159-179.
    4. repec:eee:ejores:v:274:y:2019:i:3:p:1047-1054 is not listed on IDEAS
    5. Fare, Rolf & Grosskopf, Shawna, 1985. " Nonparametric Cost Approach to Scale Efficiency," Scandinavian Journal of Economics, Wiley Blackwell, vol. 87(4), pages 594-604.
    6. Alexandra Daskovska & Léopold Simar & Sébastien Bellegem, 2010. "Forecasting the Malmquist productivity index," Journal of Productivity Analysis, Springer, vol. 33(2), pages 97-107, April.
    7. Leopold Simar & Valentin Zelenyuk, 2006. "On Testing Equality of Distributions of Technical Efficiency Scores," Econometric Reviews, Taylor & Francis Journals, vol. 25(4), pages 497-522.
    8. W. E. Diewert, 1974. "A Note on Aggregation and Elasticities of Substitution," Canadian Journal of Economics, Canadian Economics Association, vol. 7(1), pages 12-20, February.
    9. GIJBELS, Irène & MAMMEN, Enno & PARK, Byeong U. & SIMAR, Léopold, 1997. "On estimation of monotone and concave frontier functions," CORE Discussion Papers 1997031, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    10. Jati Sengupta, 1998. "Stochastic data envelopment analysis: a new approach," Applied Economics Letters, Taylor & Francis Journals, vol. 5(5), pages 287-290.
    11. Wheelock, David C & Wilson, Paul W, 1995. "Explaining Bank Failures: Deposit Insurance, Regulation, and Efficiency," The Review of Economics and Statistics, MIT Press, vol. 77(4), pages 689-700, November.
    12. repec:eee:ejores:v:267:y:2018:i:1:p:349-367 is not listed on IDEAS
    13. Adler, Nicole & Golany, Boaz, 2001. "Evaluation of deregulated airline networks using data envelopment analysis combined with principal component analysis with an application to Western Europe," European Journal of Operational Research, Elsevier, vol. 132(2), pages 260-273, July.
    14. Afriat, Sidney N, 1972. "Efficiency Estimation of Production Function," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 13(3), pages 568-598, October.
    15. Kneip, Alois & Simar, Léopold & Wilson, Paul W., 2008. "Asymptotics And Consistent Bootstraps For Dea Estimators In Nonparametric Frontier Models," Econometric Theory, Cambridge University Press, vol. 24(06), pages 1663-1697, December.
    16. Daraio, Cinzia & Simar, Léopold, 2014. "Directional distances and their robust versions: Computational and testing issues," European Journal of Operational Research, Elsevier, vol. 237(1), pages 358-369.
    17. Simar, Léopold & Wilson, Paul W., 2013. "Estimation and Inference in Nonparametric Frontier Models: Recent Developments and Perspectives," Foundations and Trends(R) in Econometrics, now publishers, vol. 5(3–4), pages 183-337, June.
    18. Ranyard, J.C. & Fildes, R. & Hu, Tun-I, 2015. "Reassessing the scope of OR practice: The Influences of Problem Structuring Methods and the Analytics Movement," European Journal of Operational Research, Elsevier, vol. 245(1), pages 1-13.
    19. William Cooper & Zhimin Huang & Vedran Lelas & Susan Li & Ole Olesen, 1998. "Chance Constrained Programming Formulations for Stochastic Characterizations of Efficiency and Dominance in DEA," Journal of Productivity Analysis, Springer, vol. 9(1), pages 53-79, January.
    20. Jeong, Seok-Oh & Simar, Léopold, 2006. "Linearly interpolated FDH efficiency score for nonconvex frontiers," Journal of Multivariate Analysis, Elsevier, vol. 97(10), pages 2141-2161, November.
    21. Simar, Léopold & Vanhems, Anne & Wilson, Paul W., 2012. "Statistical inference for DEA estimators of directional distances," European Journal of Operational Research, Elsevier, vol. 220(3), pages 853-864.
    22. Cinzia Daraio & Léopold Simar, 2005. "Introducing Environmental Variables in Nonparametric Frontier Models: a Probabilistic Approach," Journal of Productivity Analysis, Springer, vol. 24(1), pages 93-121, September.
    23. Kneip, Alois & Park, Byeong U. & Simar, Léopold, 1998. "A Note On The Convergence Of Nonparametric Dea Estimators For Production Efficiency Scores," Econometric Theory, Cambridge University Press, vol. 14(6), pages 783-793, December.
    24. Diewert, W Erwin, 1978. "Superlative Index Numbers and Consistency in Aggregation," Econometrica, Econometric Society, vol. 46(4), pages 883-900, July.
    25. Loren Tauer, 2001. "Input aggregation and computed technical efficiency," Applied Economics Letters, Taylor & Francis Journals, vol. 8(5), pages 295-297.
    26. Niels Christian Petersen, 1990. "Data Envelopment Analysis on a Relaxed Set of Assumptions," Management Science, INFORMS, vol. 36(3), pages 305-314, March.
    27. Cook, Wade D. & Ruiz, José L. & Sirvent, Inmaculada & Zhu, Joe, 2017. "Within-group common benchmarking using DEA," European Journal of Operational Research, Elsevier, vol. 256(3), pages 901-910.
    28. repec:kap:jproda:v:47:y:2017:i:3:d:10.1007_s11123-016-0474-2 is not listed on IDEAS
    29. Boussofiane, A. & Dyson, R. G. & Thanassoulis, E., 1991. "Applied data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 52(1), pages 1-15, May.
    30. Balk, Bert M., 2004. "Decompositions of Fisher indexes," Economics Letters, Elsevier, vol. 82(1), pages 107-113, January.
    31. Finn Førsund & Nikias Sarafoglou, 2002. "On the Origins of Data Envelopment Analysis," Journal of Productivity Analysis, Springer, vol. 17(1), pages 23-40, January.
    32. Adler, Nicole & Yazhemsky, Ekaterina, 2010. "Improving discrimination in data envelopment analysis: PCA-DEA or variable reduction," European Journal of Operational Research, Elsevier, vol. 202(1), pages 273-284, April.
    33. Badin, Luiza & Daraio, Cinzia & Simar, Léopold, 2010. "Optimal bandwidth selection for conditional efficiency measures: A data-driven approach," European Journal of Operational Research, Elsevier, vol. 201(2), pages 633-640, March.
    34. Park, Byeong U. & Simar, Léopold & Zelenyuk, Valentin, 2008. "Local likelihood estimation of truncated regression and its partial derivatives: Theory and application," Journal of Econometrics, Elsevier, vol. 146(1), pages 185-198, September.
    35. Léopold Simar & Paul W. Wilson, 2015. "Statistical Approaches for Non-parametric Frontier Models: A Guided Tour," International Statistical Review, International Statistical Institute, vol. 83(1), pages 77-110, April.
    36. Richard S. BARR & Lawrence M. SEIFORD & Thomas F. SIEMS, 1994. "Forecasting Bank Failure : A Non-Parametric Frontier Estimation Approach," Discussion Papers (REL - Recherches Economiques de Louvain) 1994041, Université catholique de Louvain, Institut de Recherches Economiques et Sociales (IRES).
    37. Rolf Fare & Valentin Zelenyuk, 2002. "Input aggregation and technical efficiency," Applied Economics Letters, Taylor & Francis Journals, vol. 9(10), pages 635-636.
    38. Charnes, A. & Cooper, W. W. & Rhodes, E., 1978. "Measuring the efficiency of decision making units," European Journal of Operational Research, Elsevier, vol. 2(6), pages 429-444, November.
    39. repec:wsi:ijitdm:v:04:y:2005:i:03:n:s021962200500160x is not listed on IDEAS
    40. Park, B.U. & Simar, L. & Weiner, Ch., 2000. "The Fdh Estimator For Productivity Efficiency Scores," Econometric Theory, Cambridge University Press, vol. 16(6), pages 855-877, December.
    41. Matthew Liberatore & Wenhong Luo, 2011. "INFORMS and the Analytics Movement: The View of the Membership," Interfaces, INFORMS, vol. 41(6), pages 578-589, December.
    42. Kneip, Alois & Simar, Léopold & Wilson, Paul W., 2015. "When Bias Kills The Variance: Central Limit Theorems For Dea And Fdh Efficiency Scores," Econometric Theory, Cambridge University Press, vol. 31(2), pages 394-422, April.
    43. L. V. Kantorovich, 1960. "Mathematical Methods of Organizing and Planning Production," Management Science, INFORMS, vol. 6(4), pages 366-422, July.
    44. Byeong Park & Léopold Simar & Valentin Zelenyuk, 2015. "Categorical data in local maximum likelihood: theory and applications to productivity analysis," Journal of Productivity Analysis, Springer, vol. 43(2), pages 199-214, April.
    45. Simar, Leopold & Wilson, Paul W., 2007. "Estimation and inference in two-stage, semi-parametric models of production processes," Journal of Econometrics, Elsevier, vol. 136(1), pages 31-64, January.
    46. Richard Barr & Matthew Durchholz, 1997. "Parallel and hierarchical decomposition approaches for solving large-scale Data Envelopment Analysis models," Annals of Operations Research, Springer, vol. 73(0), pages 339-372, October.
    47. Misiunas, Nicholas & Oztekin, Asil & Chen, Yao & Chandra, Kavitha, 2016. "DEANN: A healthcare analytic methodology of data envelopment analysis and artificial neural networks for the prediction of organ recipient functional status," Omega, Elsevier, vol. 58(C), pages 46-54.
    48. R. Allen & A. Athanassopoulos & R.G. Dyson & E. Thanassoulis, 1997. "Weights restrictions and value judgements in Data Envelopment Analysis: Evolution, development and future directions," Annals of Operations Research, Springer, vol. 73(0), pages 13-34, October.
    49. Wesley D. Seitz, 1970. "The Measurement of Efficiency Relative to a Frontier Production Function," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 52(4), pages 505-511.
    50. Simar, L., 1991. "Estimating efficiencies from frontier models with panel data: a comparison of parametric, non-parametric and semi-parametric methods with boot strapping," CORE Discussion Papers 1991026, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    51. Charnes, A. & Neralic, L., 1990. "Sensitivity analysis of the additive model in data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 48(3), pages 332-341, October.
    52. Cinzia Daraio & Léopold Simar, 2007. "Conditional nonparametric frontier models for convex and nonconvex technologies: a unifying approach," Journal of Productivity Analysis, Springer, vol. 28(1), pages 13-32, October.
    53. Léopold Simar & Valentin Zelenyuk, 2011. "Stochastic FDH/DEA estimators for frontier analysis," Journal of Productivity Analysis, Springer, vol. 36(1), pages 1-20, August.
    54. Rolf Fare & Shawna Grosskopf & Valentin Zelenyuk, 2004. "Aggregation bias and its bounds in measuring technical efficiency," Applied Economics Letters, Taylor & Francis Journals, vol. 11(10), pages 657-660.
    55. Stoker, Thomas M, 1993. "Empirical Approaches to the Problem of Aggregation Over Individuals," Journal of Economic Literature, American Economic Association, vol. 31(4), pages 1827-1874, December.
    56. Seitz, Wesley D, 1971. "Productive Efficiency in the Steam-Electric Generating Industry," Journal of Political Economy, University of Chicago Press, vol. 79(4), pages 878-886, July-Aug..
    57. Caves, Douglas W & Christensen, Laurits R & Diewert, W Erwin, 1982. "Multilateral Comparisons of Output, Input, and Productivity Using Superlative Index Numbers," Economic Journal, Royal Economic Society, vol. 92(365), pages 73-86, March.
    58. Léopold Simar & Ingrid Keilegom & Valentin Zelenyuk, 2017. "Nonparametric least squares methods for stochastic frontier models," Journal of Productivity Analysis, Springer, vol. 47(3), pages 189-204, June.
    59. J. v. Neumann, 1945. "A Model of General Economic Equilibrium," Review of Economic Studies, Oxford University Press, vol. 13(1), pages 1-9.
    60. Caves, Douglas W & Christensen, Laurits R & Diewert, W Erwin, 1982. "The Economic Theory of Index Numbers and the Measurement of Input, Output, and Productivity," Econometrica, Econometric Society, vol. 50(6), pages 1393-1414, November.
    61. Rajiv D. Banker & Ram Natarajan, 2008. "Evaluating Contextual Variables Affecting Productivity Using Data Envelopment Analysis," Operations Research, INFORMS, vol. 56(1), pages 48-58, February.
    62. Kao, Chiang, 2014. "Network data envelopment analysis: A review," European Journal of Operational Research, Elsevier, vol. 239(1), pages 1-16.
    63. Cook, Wade D. & Tone, Kaoru & Zhu, Joe, 2014. "Data envelopment analysis: Prior to choosing a model," Omega, Elsevier, vol. 44(C), pages 1-4.
    64. Subodh Kumar & R. Robert Russell, 2002. "Technological Change, Technological Catch-up, and Capital Deepening: Relative Contributions to Growth and Convergence," American Economic Review, American Economic Association, vol. 92(3), pages 527-548, June.
    65. O. B. Olesen & N. C. Petersen, 1995. "Chance Constrained Efficiency Evaluation," Management Science, INFORMS, vol. 41(3), pages 442-457, March.
    66. David C. Wheelock & Paul W. Wilson, 2000. "Why do Banks Disappear? The Determinants of U.S. Bank Failures and Acquisitions," The Review of Economics and Statistics, MIT Press, vol. 82(1), pages 127-138, February.
    67. Diewert, W. E., 1976. "Exact and superlative index numbers," Journal of Econometrics, Elsevier, vol. 4(2), pages 115-145, May.
    68. W. Erwin Diewert, 1980. "Aggregation Problems in the Measurement of Capital," NBER Chapters,in: The Measurement of Capital, pages 433-538 National Bureau of Economic Research, Inc.
    69. Léopold Simar & Paul Wilson, 2011. "Two-stage DEA: caveat emptor," Journal of Productivity Analysis, Springer, vol. 36(2), pages 205-218, October.
    70. Cook, Wade D. & Seiford, Larry M., 2009. "Data envelopment analysis (DEA) - Thirty years on," European Journal of Operational Research, Elsevier, vol. 192(1), pages 1-17, January.
    71. Diewert, W E, 1971. "An Application of the Shephard Duality Theorem: A Generalized Leontief Production Function," Journal of Political Economy, University of Chicago Press, vol. 79(3), pages 481-507, May-June.
    72. Peter Bogetoft & Joseph M. Tama & Jørgen Tind, 2000. "Convex Input and Output Projections of Nonconvex Production Possibility Sets," Management Science, INFORMS, vol. 46(6), pages 858-869, June.
    73. R. D. Banker & A. Charnes & W. W. Cooper, 1984. "Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis," Management Science, INFORMS, vol. 30(9), pages 1078-1092, September.
    74. Olesen, Ole B. & Petersen, Niels Christian, 2016. "Stochastic Data Envelopment Analysis—A review," European Journal of Operational Research, Elsevier, vol. 251(1), pages 2-21.
    75. Peter Bogetoft, 1996. "DEA on Relaxed Convexity Assumptions," Management Science, INFORMS, vol. 42(3), pages 457-465, March.
    76. Kumbhakar, Subal C. & Park, Byeong U. & Simar, Leopold & Tsionas, Efthymios G., 2007. "Nonparametric stochastic frontiers: A local maximum likelihood approach," Journal of Econometrics, Elsevier, vol. 137(1), pages 1-27, March.
    77. Léopold Simar & Valentin Zelenyuk, 2007. "Statistical inference for aggregates of Farrell-type efficiencies," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 22(7), pages 1367-1394.
    78. Dyson, R. G. & Allen, R. & Camanho, A. S. & Podinovski, V. V. & Sarrico, C. S. & Shale, E. A., 2001. "Pitfalls and protocols in DEA," European Journal of Operational Research, Elsevier, vol. 132(2), pages 245-259, July.
    79. Ali, Agha Iqbal, 1993. "Streamlined computation for data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 64(1), pages 61-67, January.
    80. Bădin, Luiza & Daraio, Cinzia & Simar, Léopold, 2012. "How to measure the impact of environmental factors in a nonparametric production model," European Journal of Operational Research, Elsevier, vol. 223(3), pages 818-833.
    81. Mortenson, Michael J. & Doherty, Neil F. & Robinson, Stewart, 2015. "Operational research from Taylorism to Terabytes: A research agenda for the analytics age," European Journal of Operational Research, Elsevier, vol. 241(3), pages 583-595.
    82. Cazals, Catherine & Florens, Jean-Pierre & Simar, Leopold, 2002. "Nonparametric frontier estimation: a robust approach," Journal of Econometrics, Elsevier, vol. 106(1), pages 1-25, January.
    83. Fare, Rolf & Grosskopf, Shawna & Logan, James, 1983. "The relative efficiency of Illinois electric utilities," Resources and Energy, Elsevier, vol. 5(4), pages 349-367, December.
    84. Diewert, W E, 1992. "The Measurement of Productivity," Bulletin of Economic Research, Wiley Blackwell, vol. 44(3), pages 163-198, July.
    85. Rajiv D. Banker & Richard C. Morey, 1986. "Efficiency Analysis for Exogenously Fixed Inputs and Outputs," Operations Research, INFORMS, vol. 34(4), pages 513-521, August.
    86. Emrouznejad, Ali & Parker, Barnett R. & Tavares, Gabriel, 2008. "Evaluation of research in efficiency and productivity: A survey and analysis of the first 30 years of scholarly literature in DEA," Socio-Economic Planning Sciences, Elsevier, vol. 42(3), pages 151-157, September.
    87. repec:taf:jnlbes:v:34:y:2016:i:3:p:435-456 is not listed on IDEAS
    88. Cook, Wade D. & Liang, Liang & Zhu, Joe, 2010. "Measuring performance of two-stage network structures by DEA: A review and future perspective," Omega, Elsevier, vol. 38(6), pages 423-430, December.
    89. Rajiv D. Banker & Richard C. Morey, 1986. "The Use of Categorical Variables in Data Envelopment Analysis," Management Science, INFORMS, vol. 32(12), pages 1613-1627, December.
    90. Rajiv D. Banker, 1993. "Maximum Likelihood, Consistency and Data Envelopment Analysis: A Statistical Foundation," Management Science, INFORMS, vol. 39(10), pages 1265-1273, October.
    Full references (including those not matched with items on IDEAS)

    More about this item

    Keywords

    Data Envelopment Analysis; Productivity; Efficiency; Business Analytics; Big Data;

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:qld:uqcepa:137. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (SOE IT). General contact details of provider: http://edirc.repec.org/data/decuqau.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.