IDEAS home Printed from https://ideas.repec.org/a/eee/eneeco/v143y2025ics0140988325000623.html
   My bibliography  Save this article

The by-production models for benchmarking

Author

Listed:
  • Wang, Derek D.
  • Hu, Peng
  • Ren, Yaoyao

Abstract

By-production models, as a new class of efficiency estimation models for assessing pollution-generating technologies with undesirable outputs, are becoming increasingly popular in benchmarking studies recently. Yet the performance of these models is not well understood. In this study, we investigate the accuracy of three widely-used by-production models (by-production slacks-based measure or BP-SBM, by-production directional distance function or BP-DDF, by-production Färe-Grosskopf-Lovell or BP-FGL) through Monte-Carlo simulation, accounting for different returns-to-scale, sample sizes, noise levels, and endogeneity in data. We find that the BP-FGL model achieves the highest-ranking accuracy, as measured by Kendall's ranking correlation, under all simulated scenarios, falling in the range between 0.553 and 0.758. BP-DDF model, being the predominantly used model in literature, has the worst performance and should be deprioritized in future studies. All models exhibit better accuracy for larger samples and lower noise levels. The presence of endogeneity in data degrades the model accuracy, with the degradation effect being symmetric for positive and negative endogeneity levels. The applications of by-production models are further illuminated through a case study of coal-fired power plants in the United States to highlight the importance of model selection in applying the by-production approach, a critical issue that has been ignored by existing literature.

Suggested Citation

  • Wang, Derek D. & Hu, Peng & Ren, Yaoyao, 2025. "The by-production models for benchmarking," Energy Economics, Elsevier, vol. 143(C).
  • Handle: RePEc:eee:eneeco:v:143:y:2025:i:c:s0140988325000623
    DOI: 10.1016/j.eneco.2025.108239
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0140988325000623
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.eneco.2025.108239?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Seufert, Juergen Heinz & Arjomandi, Amir & Dakpo, K. Hervé, 2017. "Evaluating airline operational performance: A Luenberger-Hicks-Moorsteen productivity indicator," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 104(C), pages 52-68.
    2. Qiwei Xie & Linda L. Zhang & Haichao Shang & Ali Emrouznejad & Yongjun Li, 2021. "Evaluating performance of super-efficiency models in ranking efficient decision-making units based on Monte Carlo simulations," Annals of Operations Research, Springer, vol. 305(1), pages 273-323, October.
    3. Bretschger, Lucas & Jo, Ara, 2024. "Complementarity between labor and energy: A firm-level analysis," Journal of Environmental Economics and Management, Elsevier, vol. 124(C).
    4. Andor, Mark A. & Parmeter, Christopher & Sommer, Stephan, 2019. "Combining uncertainty with uncertainty to get certainty? Efficiency analysis for regulation purposes," European Journal of Operational Research, Elsevier, vol. 274(1), pages 240-252.
    5. Zhu, Liyun & Schneider, Kevin & Oude Lansink, Alfons, 2023. "Economic, environmental, and social inefficiency assessment of Dutch dairy farms based on the dynamic by-production model," European Journal of Operational Research, Elsevier, vol. 311(3), pages 1134-1145.
    6. Kumbhakar, Subal C. & Badunenko, Oleg & Willox, Michael, 2022. "Do carbon taxes affect economic and environmental efficiency? The case of British Columbia’s manufacturing plants," Energy Economics, Elsevier, vol. 115(C).
    7. Mette Asmild & Jens Hougaard & Dorte Kronborg, 2013. "Do efficiency scores depend on input mix? A statistical test and empirical illustration," Annals of Operations Research, Springer, vol. 211(1), pages 37-48, December.
    8. F Pedraja-Chaparro & J Salinas-Jiménez & P Smith, 1999. "On the quality of the data envelopment analysis model," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 50(6), pages 636-644, June.
    9. Nataraja, Niranjan R. & Johnson, Andrew L., 2011. "Guidelines for using variable selection techniques in data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 215(3), pages 662-669, December.
    10. Fukuyama, Hirofumi & Tan, Yong, 2022. "Implementing strategic disposability for performance evaluation: Innovation, stability, profitability and corporate social responsibility in Chinese banking," European Journal of Operational Research, Elsevier, vol. 296(2), pages 652-668.
    11. Tim Coelli & Ludwig Lauwers & Guido Huylenbroeck, 2007. "Environmental efficiency measurement and the materials balance condition," Journal of Productivity Analysis, Springer, vol. 28(1), pages 3-12, October.
    12. Hampf, Benjamin & Rodseth, Kenneth, 2019. "Environmental Efficiency Measurement with Heterogeneous Input Quality: A Nonparametric Analysis of U.S. Power Plants," Publications of Darmstadt Technical University, Institute for Business Studies (BWL) 118700, Darmstadt Technical University, Department of Business Administration, Economics and Law, Institute for Business Studies (BWL).
    13. Jeanneaux, Philippe & Latruffe, Laure, 2016. "Modelling pollution-generating technologies in performance benchmarking: Recent developments, limits and future prospects in the nonparametric frameworkAuthor-Name: Dakpo, K. Hervé," European Journal of Operational Research, Elsevier, vol. 250(2), pages 347-359.
    14. Bellini, Tiziano, 2012. "Forward search outlier detection in data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 216(1), pages 200-207.
    15. Hampf, Benjamin, 2018. "Measuring Inefficiency in the Presence of Bad Outputs: Does the Disposability Assumption Matter?," Publications of Darmstadt Technical University, Institute for Business Studies (BWL) 110815, Darmstadt Technical University, Department of Business Administration, Economics and Law, Institute for Business Studies (BWL).
    16. Perelman, Sergio & Santín, Daniel, 2009. "How to generate regularly behaved production data? A Monte Carlo experimentation on DEA scale efficiency measurement," European Journal of Operational Research, Elsevier, vol. 199(1), pages 303-310, November.
    17. Engida, Tadesse Getacher & Rao, Xudong & Oude Lansink, Alfons G.J.M., 2020. "A dynamic by-production framework for analyzing inefficiency associated with corporate social responsibility," European Journal of Operational Research, Elsevier, vol. 287(3), pages 1170-1179.
    18. Hampf, Benjamin & Rødseth, Kenneth Løvold, 2015. "Carbon dioxide emission standards for U.S. power plants: An efficiency analysis perspective," Energy Economics, Elsevier, vol. 50(C), pages 140-153.
    19. Krüger, Jens J., 2012. "A Monte Carlo study of old and new frontier methods for efficiency measurement," European Journal of Operational Research, Elsevier, vol. 222(1), pages 137-148.
    20. Cook, Wade D. & Tone, Kaoru & Zhu, Joe, 2014. "Data envelopment analysis: Prior to choosing a model," Omega, Elsevier, vol. 44(C), pages 1-4.
    21. Huifen Chen, 2001. "Initialization for NORTA: Generation of Random Vectors with Specified Marginals and Correlations," INFORMS Journal on Computing, INFORMS, vol. 13(4), pages 312-331, November.
    22. Shi, Xiao & Wang, Libo & Emrouznejad, Ali, 2023. "Performance evaluation of Chinese commercial banks by an improved slacks-based DEA model," Socio-Economic Planning Sciences, Elsevier, vol. 90(C).
    23. Hoang, Viet-Ngu & Coelli, Tim, 2011. "Measurement of agricultural total factor productivity growth incorporating environmental factors: A nutrients balance approach," Journal of Environmental Economics and Management, Elsevier, vol. 62(3), pages 462-474.
    24. Kumbhakar, Subal C. & Tsionas, Efthymios G., 2016. "The good, the bad and the technology: Endogeneity in environmental production models," Journal of Econometrics, Elsevier, vol. 190(2), pages 315-327.
    25. Fare, Rolf, 1989. "Multilateral Productivity Comparisons When Some Outputs Are Undesirable: A Nonparametric Approach," The Review of Economics and Statistics, MIT Press, vol. 71(1), pages 90-98, February.
    26. Herbert Lewis & Thomas Sexton, 2004. "Data Envelopment Analysis with Reverse Inputs and Outputs," Journal of Productivity Analysis, Springer, vol. 21(2), pages 113-132, March.
    27. Dariush Khezrimotlagh & Wade D. Cook & Joe Zhu, 2021. "Number of performance measures versus number of decision making units in DEA," Annals of Operations Research, Springer, vol. 303(1), pages 529-562, August.
    28. Jürgen Bitzer & Erkan Gören, 2016. "Measuring capital services by energy use: an empirical comparative study," Applied Economics, Taylor & Francis Journals, vol. 48(53), pages 5152-5167, November.
    29. Arjomandi, Amir & Dakpo, K. Hervé & Seufert, Juergen Heinz, 2018. "Have Asian airlines caught up with European Airlines? A by-production efficiency analysis," Transportation Research Part A: Policy and Practice, Elsevier, vol. 116(C), pages 389-403.
    30. Zelenyuk, Valentin, 2020. "Aggregation of inputs and outputs prior to Data Envelopment Analysis under big data," European Journal of Operational Research, Elsevier, vol. 282(1), pages 172-187.
    31. Rajiv D. Banker & Hsihui Chang & Zhiqiang Zheng, 2017. "On the use of super-efficiency procedures for ranking efficient units and identifying outliers," Annals of Operations Research, Springer, vol. 250(1), pages 21-35, March.
    32. Moody, Carlisle E, Jr, 1974. "The Measurement of Capital Services by Electrical Energy," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 36(1), pages 45-52, February.
    33. Banker, Rajiv D. & Chang, Hsihui & Cooper, William W., 2004. "A simulation study of DEA and parametric frontier models in the presence of heteroscedasticity," European Journal of Operational Research, Elsevier, vol. 153(3), pages 624-640, March.
    34. Chien-Ming Chen & Magali A. Delmas, 2012. "Measuring Eco-Inefficiency: A New Frontier Approach," Operations Research, INFORMS, vol. 60(5), pages 1064-1079, October.
    35. Fare, Rolf & Grosskopf, Shawna, 2004. "Modeling undesirable factors in efficiency evaluation: Comment," European Journal of Operational Research, Elsevier, vol. 157(1), pages 242-245, August.
    36. Cordero, José Manuel & Santín, Daniel & Sicilia, Gabriela, 2015. "Testing the accuracy of DEA estimates under endogeneity through a Monte Carlo simulation," European Journal of Operational Research, Elsevier, vol. 244(2), pages 511-518.
    37. Benjamin Hampf, 2018. "Measuring inefficiency in the presence of bad outputs: Does the disposability assumption matter?," Empirical Economics, Springer, vol. 54(1), pages 101-127, February.
    38. Wang, Derek D. & Ren, Yaoyao, 2024. "Accuracy of Deterministic Nonparametric Frontier Models with Undesirable Outputs," European Journal of Operational Research, Elsevier, vol. 315(2), pages 596-612.
    39. Atakelty Hailu & Terrence S. Veeman, 2001. "Non-parametric Productivity Analysis with Undesirable Outputs: An Application to the Canadian Pulp and Paper Industry," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 83(3), pages 605-616.
    40. Ayres, Robert U & Kneese, Allen V, 1969. "Production , Consumption, and Externalities," American Economic Review, American Economic Association, vol. 59(3), pages 282-297, June.
    41. Qiwei Xie & Linda Zhang & Haichao Shang & Ali Emrouznejad & Yongjun Li, 2021. "Evaluating performance of super-efficiency models in ranking efficient decision-making units based on Monte Carlo simulations," Post-Print hal-03604012, HAL.
    42. Chih‐Ching Yang & Ching‐Kai Hsiao & Ming‐Miin Yu, 2008. "Technical efficiency and impact of environmental regulations in farrow‐to‐finish swine production in Taiwan," Agricultural Economics, International Association of Agricultural Economists, vol. 39(1), pages 51-61, July.
    43. Cui, Qiang & Li, Ye, 2017. "Airline efficiency measures under CNG2020 strategy: An application of a Dynamic By-production model," Transportation Research Part A: Policy and Practice, Elsevier, vol. 106(C), pages 130-143.
    44. Centorrino, Samuele & Parmeter, Christopher F., 2024. "Nonparametric estimation of stochastic frontier models with weak separability," Journal of Econometrics, Elsevier, vol. 238(2).
    45. Murty, Sushama & Robert Russell, R. & Levkoff, Steven B., 2012. "On modeling pollution-generating technologies," Journal of Environmental Economics and Management, Elsevier, vol. 64(1), pages 117-135.
    46. Banker, Rajiv D. & Gadh, Vandana M. & Gorr, Wilpen L., 1993. "A Monte Carlo comparison of two production frontier estimation methods: Corrected ordinary least squares and data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 67(3), pages 332-343, June.
    47. Holland, D. S. & Lee, S. T., 2002. "Impacts of random noise and specification on estimates of capacity derived from data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 137(1), pages 10-21, February.
    48. Kumbhakar, Subal C. & Badunenko, Oleg & Willox, Michael, 2025. "Corrigendum to “Do carbon taxes affect economic and environmental efficiency? The case of British Columbia's manufacturing plants” [Energy Economics Volume 115, November 2022, 106359]," Energy Economics, Elsevier, vol. 144(C).
    49. Welch, Eric & Barnum, Darold, 2009. "Joint environmental and cost efficiency analysis of electricity generation," Ecological Economics, Elsevier, vol. 68(8-9), pages 2336-2343, June.
    50. Rolf Färe & Shawna Grosskopf, 2003. "Nonparametric Productivity Analysis with Undesirable Outputs: Comment," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 85(4), pages 1070-1074.
    51. A. Charnes & W. W. Cooper, 1962. "Programming with linear fractional functionals," Naval Research Logistics Quarterly, John Wiley & Sons, vol. 9(3‐4), pages 181-186, September.
    52. Deng, Haiyan & Zheng, Wangyi & Shen, Zhiyang & Štreimikienė, Dalia, 2023. "Does fiscal expenditure promote green agricultural productivity gains: An investigation on corn production," Applied Energy, Elsevier, vol. 334(C).
    53. Shen, Zhiyang & Baležentis, Tomas & Chen, Xueli & Valdmanis, Vivian, 2018. "Green growth and structural change in Chinese agricultural sector during 1997–2014," China Economic Review, Elsevier, vol. 51(C), pages 83-96.
    54. Seiford, Lawrence M. & Zhu, Joe, 2002. "Modeling undesirable factors in efficiency evaluation," European Journal of Operational Research, Elsevier, vol. 142(1), pages 16-20, October.
    55. Hampf, Benjamin & Rødseth, Kenneth Løvold, 2019. "Environmental efficiency measurement with heterogeneous input quality: A nonparametric analysis of U.S. power plants," Energy Economics, Elsevier, vol. 81(C), pages 610-625.
    56. Juan Aparicio & Magdalena Kapelko & Lidia Ortiz, 2021. "Modelling environmental inefficiency under a quota system," Operational Research, Springer, vol. 21(2), pages 1097-1124, June.
    57. Aigner, Dennis & Lovell, C. A. Knox & Schmidt, Peter, 1977. "Formulation and estimation of stochastic frontier production function models," Journal of Econometrics, Elsevier, vol. 6(1), pages 21-37, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wang, Derek D. & Ren, Yaoyao, 2024. "Accuracy of Deterministic Nonparametric Frontier Models with Undesirable Outputs," European Journal of Operational Research, Elsevier, vol. 315(2), pages 596-612.
    2. Andreas Eder, 2022. "Environmental efficiency measurement when producers control pollutants under heterogeneous conditions: a generalization of the materials balance approach," Journal of Productivity Analysis, Springer, vol. 57(2), pages 157-176, April.
    3. Andreas Eder, 2021. "Environmental efficiency measurement when producers control pollutants under heterogeneous conditions: a generalization of the materials balance approach," Working Papers 752021, University of Natural Resources and Life Sciences, Vienna, Department of Economics and Social Sciences, Institute for Sustainable Economic Development.
    4. repec:zbw:inwedp:752021 is not listed on IDEAS
    5. Hampf, Benjamin & Rødseth, Kenneth Løvold, 2019. "Environmental efficiency measurement with heterogeneous input quality: A nonparametric analysis of U.S. power plants," Energy Economics, Elsevier, vol. 81(C), pages 610-625.
    6. Wang, Ke & Wei, Yi-Ming & Huang, Zhimin, 2018. "Environmental efficiency and abatement efficiency measurements of China's thermal power industry: A data envelopment analysis based materials balance approach," European Journal of Operational Research, Elsevier, vol. 269(1), pages 35-50.
    7. Niu, Yiran & Boussemart, Jean-Philippe & Shen, Zhiyang & Vardanyan, Michael, 2024. "Performance evaluation using multi-stage production frameworks: Assessing the tradeoffs among the economic, environmental, and social well-being," European Journal of Operational Research, Elsevier, vol. 318(3), pages 1000-1013.
    8. Ke Wang & Yi-Ming Wei & Zhimin Huang, 2017. "Environmental efficiency and abatement efficiency measurements of China¡¯s thermal power industry: A data envelopment analysis based materials balance approach," CEEP-BIT Working Papers 108, Center for Energy and Environmental Policy Research (CEEP), Beijing Institute of Technology.
    9. Dakpo, Hervé K & Jeanneaux, Philippe & Latruffe, Laure, 2014. "Inclusion of undesirable outputs in production technology modeling: The case of greenhouse gas emissions in French meat sheep farming," Working Papers 207806, Institut National de la recherche Agronomique (INRA), Departement Sciences Sociales, Agriculture et Alimentation, Espace et Environnement (SAE2).
    10. Aparicio, Juan & Kapelko, Magdalena & Zofío, José L., 2020. "The measurement of environmental economic inefficiency with pollution-generating technologies," Resource and Energy Economics, Elsevier, vol. 62(C).
    11. Jeanneaux, Philippe & Latruffe, Laure, 2016. "Modelling pollution-generating technologies in performance benchmarking: Recent developments, limits and future prospects in the nonparametric frameworkAuthor-Name: Dakpo, K. Hervé," European Journal of Operational Research, Elsevier, vol. 250(2), pages 347-359.
    12. Guillen, Maria D. & Aparicio, Juan & Kapelko, Magdalena & Esteve, Miriam, 2025. "Measuring environmental inefficiency through machine learning: An approach based on efficiency analysis trees and by-production technology," European Journal of Operational Research, Elsevier, vol. 321(2), pages 529-542.
    13. Abad, Arnaud & Briec, Walter, 2019. "On the axiomatic of pollution-generating technologies: Non-parametric production analysis," European Journal of Operational Research, Elsevier, vol. 277(1), pages 377-390.
    14. Ke Wang & Zhifu Mi & Yi‐Ming Wei, 2019. "Will Pollution Taxes Improve Joint Ecological and Economic Efficiency of Thermal Power Industry in China?: A DEA‐Based Materials Balance Approach," Journal of Industrial Ecology, Yale University, vol. 23(2), pages 389-401, April.
    15. Yu, Ming-Miin & Rakshit, Ipsita, 2023. "Target setting for airlines incorporating CO2 emissions: The DEA bargaining approach," Journal of Air Transport Management, Elsevier, vol. 108(C).
    16. Lee, Chia-Yen, 2018. "Mixed-strategy Nash equilibrium in data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 266(3), pages 1013-1024.
    17. Atkinson, Scott E. & Tsionas, Mike G., 2021. "Generalized estimation of productivity with multiple bad outputs: The importance of materials balance constraints," European Journal of Operational Research, Elsevier, vol. 292(3), pages 1165-1186.
    18. Rødseth, Kenneth Løvold, 2023. "Shadow pricing of electricity generation using stochastic and deterministic materials balance models," Applied Energy, Elsevier, vol. 341(C).
    19. Yuanxiang Zhou & Shan Wang & Shuqi Xu & Qingyuan Zhu, 2025. "Big data in data envelopment analysis with undesirable outputs based on simulation and environmental-health matching data of Chinese industrial enterprises," Annals of Operations Research, Springer, vol. 348(1), pages 279-298, May.
    20. Benjamin Hampf, 2018. "Measuring inefficiency in the presence of bad outputs: Does the disposability assumption matter?," Empirical Economics, Springer, vol. 54(1), pages 101-127, February.
    21. Fang, Lei, 2020. "Opening the “black box” of environmental production technology in a nonparametric analysis," European Journal of Operational Research, Elsevier, vol. 286(2), pages 769-780.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:eneeco:v:143:y:2025:i:c:s0140988325000623. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eneco .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.