IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v240y2015i2p518-527.html
   My bibliography  Save this article

The impact of mismeasurement in performance benchmarking: A Monte Carlo comparison of SFA and DEA with different multi-period budgeting strategies

Author

Listed:
  • Oh, Seog-Chan
  • Shin, Jaemin

Abstract

Performance-based budgeting has received increasing attention from public and for-profit organizations in an effort to achieve a fair and balanced allocation of funds among their individual producers or operating units for overall system optimization. Although existing frontier estimation models can be used to measure and rank the performance of each producer, few studies have addressed how the mismeasurement by frontier estimation models affects the budget allocation and system performance. There is therefore a need for analysis of the accuracy of performance assessments in performance-based budgeting. This paper reports the results of a Monte Carlo analysis in which measurement errors are introduced and the system throughput in various experimental scenarios is compared. Each scenario assumes a different multi-period budgeting strategy and production frontier estimation model; the frontier estimation models considered are stochastic frontier analysis (SFA) and data envelopment analysis (DEA). The main results are as follows: (1) the selection of a proper budgeting strategy and benchmark model can lead to substantial improvement in the system throughput; (2) a “peanut butter” strategy outperforms a discriminative strategy in the presence of relatively high measurement errors, but a discriminative strategy is preferred for small measurement errors; (3) frontier estimation models outperform models with randomly-generated ranks even in cases with relatively high measurement errors; (4) SFA outperforms DEA for small measurement errors, but DEA becomes increasingly favorable relative to SFA as the measurement errors increase.

Suggested Citation

  • Oh, Seog-Chan & Shin, Jaemin, 2015. "The impact of mismeasurement in performance benchmarking: A Monte Carlo comparison of SFA and DEA with different multi-period budgeting strategies," European Journal of Operational Research, Elsevier, vol. 240(2), pages 518-527.
  • Handle: RePEc:eee:ejores:v:240:y:2015:i:2:p:518-527
    DOI: 10.1016/j.ejor.2014.07.026
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S037722171400592X
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ejor.2014.07.026?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Gong, Byeong-Ho & Sickles, Robin C., 1992. "Finite sample evidence on the performance of stochastic frontiers and data envelopment analysis using panel data," Journal of Econometrics, Elsevier, vol. 51(1-2), pages 259-284.
    2. Ruggiero, John, 1999. "Efficiency estimation and error decomposition in the stochastic frontier model: A Monte Carlo analysis," European Journal of Operational Research, Elsevier, vol. 115(3), pages 555-563, June.
    3. Banker, Rajiv D. & Gadh, Vandana M. & Gorr, Wilpen L., 1993. "A Monte Carlo comparison of two production frontier estimation methods: Corrected ordinary least squares and data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 67(3), pages 332-343, June.
    4. Yan, Hong & Wei, Quanling & Hao, Gang, 2002. "DEA models for resource reallocation and production input/output estimation," European Journal of Operational Research, Elsevier, vol. 136(1), pages 19-31, January.
    5. Pekka Korhonen & Mikko Syrjänen, 2004. "Resource Allocation Based on Efficiency Analysis," Management Science, INFORMS, vol. 50(8), pages 1134-1144, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Miao, Chenglin & Fang, Debin & Sun, Liyan & Luo, Qiaoling, 2017. "Natural resources utilization efficiency under the influence of green technological innovation," Resources, Conservation & Recycling, Elsevier, vol. 126(C), pages 153-161.
    2. Ahn, Heinz & Clermont, Marcel & Langner, Julia, 2023. "Comparative performance analysis of frontier-based efficiency measurement methods – A Monte Carlo simulation," European Journal of Operational Research, Elsevier, vol. 307(1), pages 294-312.
    3. Marcos Gonçalves Perroni & Claudimar Pereira da Veiga & Zhaohui Su & Fernando Maciel Ramos & Wesley Vieira da Silva, 2023. "Dynamic Equilibrium of Sustainable Ecosystem Variables: An Experiment," Sustainability, MDPI, vol. 15(8), pages 1-21, April.
    4. Tsionas, Mike G., 2023. "Performance estimation when the distribution of inefficiency is unknown," European Journal of Operational Research, Elsevier, vol. 304(3), pages 1212-1222.
    5. Zaiwu Gong & Xiaoqing Chen, 2017. "Analysis of Interval Data Envelopment Efficiency Model Considering Different Distribution Characteristics—Based on Environmental Performance Evaluation of the Manufacturing Industry," Sustainability, MDPI, vol. 9(12), pages 1-25, November.
    6. Zetian Yu & Hao Liu & Hua Peng & Qiantong Xia & Xiaoxia Dong, 2023. "Production Efficiency of Raw Milk and Its Determinants: Application of Combining Data Envelopment Analysis and Stochastic Frontier Analysis," Agriculture, MDPI, vol. 13(2), pages 1-25, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mark Andor & Frederik Hesse, "undated". "A Monte Carlo Simulation comparing DEA, SFA and two simple approaches to combine efficiency estimates," Working Papers 201177, Institute of Spatial and Housing Economics, Munster Universitary.
    2. Mark Andor & Frederik Hesse, "undated". "The StoNED age: The Departure Into a New Era of Efficiency Analysis? An MC study Comparing StoNED and the "Oldies" (SFA and DEA)," Working Papers 201285, Institute of Spatial and Housing Economics, Munster Universitary.
    3. John Ruggiero, 2004. "Data envelopment analysis with stochastic data," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 55(9), pages 1008-1012, September.
    4. Julia Schaefer & Marcel Clermont, 2018. "Stochastic non-smooth envelopment of data for multi-dimensional output," Journal of Productivity Analysis, Springer, vol. 50(3), pages 139-154, December.
    5. Mark Andor & Frederik Hesse, 2014. "The StoNED age: the departure into a new era of efficiency analysis? A monte carlo comparison of StoNED and the “oldies” (SFA and DEA)," Journal of Productivity Analysis, Springer, vol. 41(1), pages 85-109, February.
    6. Krüger, Jens J., 2012. "A Monte Carlo study of old and new frontier methods for efficiency measurement," European Journal of Operational Research, Elsevier, vol. 222(1), pages 137-148.
    7. Ruggiero, John, 2003. "Comment on estimating school efficiency," Economics of Education Review, Elsevier, vol. 22(6), pages 631-634, December.
    8. Collier, Trevor & Johnson, Andrew L. & Ruggiero, John, 2011. "Technical efficiency estimation with multiple inputs and multiple outputs using regression analysis," European Journal of Operational Research, Elsevier, vol. 208(2), pages 153-160, January.
    9. Ondrich, Jan & Ruggiero, John, 2001. "Efficiency measurement in the stochastic frontier model," European Journal of Operational Research, Elsevier, vol. 129(2), pages 434-442, March.
    10. Ruggiero, John, 2006. "Measurement error, education production and data envelopment analysis," Economics of Education Review, Elsevier, vol. 25(3), pages 327-333, June.
    11. Uwe Jensen, 2005. "Misspecification Preferred: The Sensitivity of Inefficiency Rankings," Journal of Productivity Analysis, Springer, vol. 23(2), pages 223-244, May.
    12. Ahn, Heinz & Clermont, Marcel & Langner, Julia, 2023. "Comparative performance analysis of frontier-based efficiency measurement methods – A Monte Carlo simulation," European Journal of Operational Research, Elsevier, vol. 307(1), pages 294-312.
    13. Ane Elixabete Ripoll-Zarraga & Sebastián Lozano, 2020. "A centralised DEA approach to resource reallocation in Spanish airports," Annals of Operations Research, Springer, vol. 288(2), pages 701-732, May.
    14. Isabel Narbón-Perpiñá & Maria Teresa Balaguer-Coll & Marko Petrović & Emili Tortosa-Ausina, 2020. "Which estimator to measure local governments’ cost efficiency? The case of Spanish municipalities," SERIEs: Journal of the Spanish Economic Association, Springer;Spanish Economic Association, vol. 11(1), pages 51-82, March.
    15. Andor, Mark A. & Parmeter, Christopher & Sommer, Stephan, 2019. "Combining uncertainty with uncertainty to get certainty? Efficiency analysis for regulation purposes," European Journal of Operational Research, Elsevier, vol. 274(1), pages 240-252.
    16. Djedje Hermann Yohou, 2015. "In Search of Fiscal Space in Africa: The Role of the Quality of Government Spending," CERDI Working papers halshs-01222812, HAL.
    17. Resti, Andrea, 2000. "Efficiency measurement for multi-product industries: A comparison of classic and recent techniques based on simulated data," European Journal of Operational Research, Elsevier, vol. 121(3), pages 559-578, March.
    18. John Ruggiero & Donald F. Vitaliano, 1999. "Assessing The Efficiency Of Public Schools Using Data Envelopment Analysis And Frontier Regression," Contemporary Economic Policy, Western Economic Association International, vol. 17(3), pages 321-331, July.
    19. HOSSEINZADEH LOTFI, Farhad & HATAMI-MARBINI, Adel & AGRELL, Per & GHOLAMI, Kobra, 2013. "Centralized resource reduction and target setting under DEA control," LIDAM Discussion Papers CORE 2013005, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    20. Yun Zhang & Robert Bartels, 1998. "The Effect of Sample Size on the Mean Efficiency in DEA with an Application to Electricity Distribution in Australia, Sweden and New Zealand," Journal of Productivity Analysis, Springer, vol. 9(3), pages 187-204, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:240:y:2015:i:2:p:518-527. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.