IDEAS home Printed from https://ideas.repec.org/a/eee/tefoso/v111y2016icp235-243.html
   My bibliography  Save this article

A novel ranking procedure for forecasting approaches using Data Envelopment Analysis

Author

Listed:
  • Emrouznejad, Ali
  • Rostami-Tabar, Bahman
  • Petridis, Konstantinos

Abstract

To compare the accuracy of different forecasting approaches an error measure is required. Many error measures have been proposed in the literature, however in practice there are some situations where different measures yield different decisions on forecasting approach selection and there is no agreement on which approach should be used. Generally forecasting measures represent ratios or percentages providing an overall image of how well fitted the forecasting technique is to the observations. This paper proposes a multiplicative Data Envelopment Analysis (DEA) model in order to rank several forecasting techniques. We demonstrate the proposed model by applying it to the set of yearly time series of the M3 competition. The usefulness of the proposed approach has been tested using the M3-competition where five error measures have been applied in and aggregated to a single DEA score.

Suggested Citation

  • Emrouznejad, Ali & Rostami-Tabar, Bahman & Petridis, Konstantinos, 2016. "A novel ranking procedure for forecasting approaches using Data Envelopment Analysis," Technological Forecasting and Social Change, Elsevier, vol. 111(C), pages 235-243.
  • Handle: RePEc:eee:tefoso:v:111:y:2016:i:c:p:235-243
    DOI: 10.1016/j.techfore.2016.07.004
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0040162516301482
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.techfore.2016.07.004?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Athanasopoulos, George & Hyndman, Rob J., 2011. "The value of feedback in forecasting competitions," International Journal of Forecasting, Elsevier, vol. 27(3), pages 845-849.
    2. B. Hollingsworth & P. Smith, 2003. "Use of ratios in data envelopment analysis," Applied Economics Letters, Taylor & Francis Journals, vol. 10(11), pages 733-735.
    3. Yokuma, J. Thomas & Armstrong, J. Scott, 1995. "Beyond accuracy: Comparison of criteria used to select forecasting methods," International Journal of Forecasting, Elsevier, vol. 11(4), pages 591-597, December.
    4. Charnes, A. & Cooper, W. W. & Rhodes, E., 1978. "Measuring the efficiency of decision making units," European Journal of Operational Research, Elsevier, vol. 2(6), pages 429-444, November.
    5. Collopy, Fred & Armstrong, J. Scott, 1992. "Expert opinions about extrapolation and the mystery of the overlooked discontinuities," International Journal of Forecasting, Elsevier, vol. 8(4), pages 575-582, December.
    6. Fildes, Robert & Wei, Yingqi & Ismail, Suzilah, 2011. "Evaluating the forecasting performance of econometric models of air passenger traffic flows using multiple error measures," International Journal of Forecasting, Elsevier, vol. 27(3), pages 902-922, July.
    7. Syntetos, Aris A. & Boylan, John E., 2005. "The accuracy of intermittent demand estimates," International Journal of Forecasting, Elsevier, vol. 21(2), pages 303-314.
    8. Chen, Yao, 2005. "Measuring super-efficiency in DEA in the presence of infeasibility," European Journal of Operational Research, Elsevier, vol. 161(2), pages 545-551, March.
    9. Lim, Dong-Joon & Anderson, Timothy R. & Inman, Oliver Lane, 2014. "Choosing effective dates from multiple optima in Technology Forecasting using Data Envelopment Analysis (TFDEA)," Technological Forecasting and Social Change, Elsevier, vol. 88(C), pages 91-97.
    10. Cook, Wade D. & Seiford, Larry M., 2009. "Data envelopment analysis (DEA) - Thirty years on," European Journal of Operational Research, Elsevier, vol. 192(1), pages 1-17, January.
    11. Davydenko, Andrey & Fildes, Robert, 2013. "Measuring forecasting accuracy: The case of judgmental adjustments to SKU-level demand forecasts," International Journal of Forecasting, Elsevier, vol. 29(3), pages 510-522.
    12. Emrouznejad, Ali & De Witte, Kristof, 2010. "COOPER-framework: A unified process for non-parametric projects," European Journal of Operational Research, Elsevier, vol. 207(3), pages 1573-1586, December.
    13. Emrouznejad, A. & De Witte, K., 2010. "COOPER-framework: A Unified Standard Process for Non-parametric Projects," Working Papers 18, Top Institute for Evidence Based Education Research.
    14. Rajiv D. Banker & Ajay Maindiratta, 1986. "Erratum to: "Piecewise Loglinear Estimation of Efficient Production Surfaces"," Management Science, INFORMS, vol. 32(3), pages 385-385, March.
    15. Charnes, A. & Cooper, W. W. & Seiford, L. & Stutz, J., 1982. "A multiplicative model for efficiency analysis," Socio-Economic Planning Sciences, Elsevier, vol. 16(5), pages 223-224.
    16. De Gooijer, Jan G. & Hyndman, Rob J., 2006. "25 years of time series forecasting," International Journal of Forecasting, Elsevier, vol. 22(3), pages 443-473.
    17. Hyndman, Rob J. & Koehler, Anne B., 2006. "Another look at measures of forecast accuracy," International Journal of Forecasting, Elsevier, vol. 22(4), pages 679-688.
    18. B. Xu & J. Ouenniche, 2011. "A multidimensional framework for performance evaluation of forecasting models: context-dependent DEA," Applied Financial Economics, Taylor & Francis Journals, vol. 21(24), pages 1873-1890, December.
    19. Makridakis, Spyros & Hibon, Michele, 2000. "The M3-Competition: results, conclusions and implications," International Journal of Forecasting, Elsevier, vol. 16(4), pages 451-476.
    20. Geurts, Michael D. & Patrick Kelly, J., 1986. "Forecasting retail sales using alternative models," International Journal of Forecasting, Elsevier, vol. 2(3), pages 261-272.
    21. W D Cook & L Liang & Y Zha & J Zhu, 2009. "A modified super-efficiency DEA model for infeasibility," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 60(2), pages 276-281, February.
    22. R. D. Banker & A. Charnes & W. W. Cooper, 1984. "Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis," Management Science, INFORMS, vol. 30(9), pages 1078-1092, September.
    23. Rajiv D. Banker & Ajay Maindiratta, 1986. "Piecewise Loglinear Estimation of Efficient Production Surfaces," Management Science, INFORMS, vol. 32(1), pages 126-135, January.
    24. Seiford, Lawrence M. & Zhu, Joe, 2002. "Modeling undesirable factors in efficiency evaluation," European Journal of Operational Research, Elsevier, vol. 142(1), pages 16-20, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Azarnoosh Kafi & Behrouz Daneshian & Mohsen Rostamy-Malkhalifeh, 2021. "Forecasting the confidence interval of efficiency in fuzzy DEA," Operations Research and Decisions, Wroclaw University of Science and Technology, Faculty of Management, vol. 31(1), pages 41-59.
    2. Georgios Digkas & Konstantinos Petridis & Alexander Chatzigeorgiou & Emmanouil Stiakakis & Ali Emrouznejad, 2020. "Measuring Spatio-temporal Efficiency: An R Implementation for Time-Evolving Units," Computational Economics, Springer;Society for Computational Economics, vol. 56(4), pages 843-864, December.
    3. Wu, Yunna & Ke, Yiming & Zhang, Ting & Liu, Fangtong & Wang, Jing, 2018. "Performance efficiency assessment of photovoltaic poverty alleviation projects in China: A three-phase data envelopment analysis model," Energy, Elsevier, vol. 159(C), pages 599-610.
    4. Azarnoosh Kafi & Behrouz Daneshian & Mohsen Rostamy-Malkhalifeh, 2021. "Forecasting the confidence interval of efficiency in fuzzy DEA," Operations Research and Decisions, Wroclaw University of Science Technology, Faculty of Management, vol. 31, pages 41-59.
    5. George Drogalas & Konstantinos Petridis & Nikolaos E. Petridis & Eleni Zografidou, 2020. "Valuation of the internal audit mechanisms in the decision support department of the local government organizations using mathematical programming," Annals of Operations Research, Springer, vol. 294(1), pages 267-280, November.
    6. Konstantinos Petridis & Georgios Drogalas & Eleni Zografidou, 2021. "Internal auditor selection using a TOPSIS/non-linear programming model," Annals of Operations Research, Springer, vol. 296(1), pages 513-539, January.
    7. Markus Vogl, 2022. "Quantitative modelling frontiers: a literature review on the evolution in financial and risk modelling after the financial crisis (2008–2019)," SN Business & Economics, Springer, vol. 2(12), pages 1-69, December.
    8. Li, Jiaxin & Wang, Zihan & Cheng, Xin & Shuai, Jing & Shuai, Chuanmin & Liu, Jing, 2020. "Has solar PV achieved the national poverty alleviation goals? Empirical evidence from the performances of 52 villages in rural China," Energy, Elsevier, vol. 201(C).
    9. Zhu, Qingyuan & Li, Xingchen & Li, Feng & Wu, Jie & Zhou, Dequn, 2020. "Energy and environmental efficiency of China's transportation sectors under the constraints of energy consumption and environmental pollutions," Energy Economics, Elsevier, vol. 89(C).
    10. Kim, Juram & Hong, Suckwon & Kang, Yubin & Lee, Changyong, 2023. "Domain-specific valuation of university technologies using bibliometrics, Jonckheere–Terpstra tests, and data envelopment analysis," Technovation, Elsevier, vol. 122(C).
    11. Mohammad Izadikhah & Elnaz Azadi & Majid Azadi & Reza Farzipoor Saen & Mehdi Toloo, 2022. "Developing a new chance constrained NDEA model to measure performance of sustainable supply chains," Annals of Operations Research, Springer, vol. 316(2), pages 1319-1347, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ghasemi, M.-R. & Ignatius, Joshua & Emrouznejad, Ali, 2014. "A bi-objective weighted model for improving the discrimination power in MCDEA," European Journal of Operational Research, Elsevier, vol. 233(3), pages 640-650.
    2. Wai‐Peng Wong & Qiang Deng & Ming-Lang Tseng & Loo‐Hay Lee & Chee‐Wooi Hooy, 2014. "A Stochastic Setting To Bank Financial Performance For Refining Efficiency Estimates," Intelligent Systems in Accounting, Finance and Management, John Wiley & Sons, Ltd., vol. 21(4), pages 225-245, October.
    3. Hatami-Marbini, Adel & Emrouznejad, Ali & Tavana, Madjid, 2011. "A taxonomy and review of the fuzzy data envelopment analysis literature: Two decades in the making," European Journal of Operational Research, Elsevier, vol. 214(3), pages 457-472, November.
    4. Banker, Rajiv D. & Cooper, William W. & Seiford, Lawrence M. & Thrall, Robert M. & Zhu, Joe, 2004. "Returns to scale in different DEA models," European Journal of Operational Research, Elsevier, vol. 154(2), pages 345-362, April.
    5. Victoria Wojcik & Harald Dyckhoff & Marcel Clermont, 2019. "Is data envelopment analysis a suitable tool for performance measurement and benchmarking in non-production contexts?," Business Research, Springer;German Academic Association for Business Research, vol. 12(2), pages 559-595, December.
    6. Marinko Škare & Danijela Rabar, 2016. "Measuring Economic Growth Using Data Envelopment Analysis," The AMFITEATRU ECONOMIC journal, Academy of Economic Studies - Bucharest, Romania, vol. 18(42), pages 386-386, May.
    7. Luis R. Murillo‐Zamorano, 2004. "Economic Efficiency and Frontier Techniques," Journal of Economic Surveys, Wiley Blackwell, vol. 18(1), pages 33-77, February.
    8. An, Qingxian & Tao, Xiangyang & Xiong, Beibei & Chen, Xiaohong, 2022. "Frontier-based incentive mechanisms for allocating common revenues or fixed costs," European Journal of Operational Research, Elsevier, vol. 302(1), pages 294-308.
    9. M. Zarepisheh & E. Khorram & G. Jahanshahloo, 2010. "Returns to scale in multiplicative models in data envelopment analysis," Annals of Operations Research, Springer, vol. 173(1), pages 195-206, January.
    10. Azarnoosh Kafi & Behrouz Daneshian & Mohsen Rostamy-Malkhalifeh, 2021. "Forecasting the confidence interval of efficiency in fuzzy DEA," Operations Research and Decisions, Wroclaw University of Science and Technology, Faculty of Management, vol. 31(1), pages 41-59.
    11. Zanella, Andreia & Camanho, Ana S. & Dias, Teresa G., 2015. "Undesirable outputs and weighting schemes in composite indicators based on data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 245(2), pages 517-530.
    12. Ravelojaona, Paola, 2019. "On constant elasticity of substitution – Constant elasticity of transformation Directional Distance Functions," European Journal of Operational Research, Elsevier, vol. 272(2), pages 780-791.
    13. Kourentzes, Nikolaos & Petropoulos, Fotios & Trapero, Juan R., 2014. "Improving forecasting by estimating time series structural components across multiple frequencies," International Journal of Forecasting, Elsevier, vol. 30(2), pages 291-302.
    14. Philippe St-Aubin & Bruno Agard, 2022. "Precision and Reliability of Forecasts Performance Metrics," Forecasting, MDPI, vol. 4(4), pages 1-22, October.
    15. Walter Briec & Laurent Cavaignac & Kristiaan Kerstens, 2020. "Input Efficiency Measures: A Generalized, Encompassing Formulation," Operations Research, INFORMS, vol. 68(6), pages 1836-1849, November.
    16. Pham, Manh D. & Zelenyuk, Valentin, 2019. "Weak disposability in nonparametric production analysis: A new taxonomy of reference technology sets," European Journal of Operational Research, Elsevier, vol. 274(1), pages 186-198.
    17. Zervopoulos, Panagiotis & Emrouznejad, Ali & Sklavos, Sokratis, 2019. "A Bayesian approach for correcting bias of data envelopment analysis estimators," MPRA Paper 91886, University Library of Munich, Germany.
    18. Fernández, David & Pozo, Carlos & Folgado, Rubén & Jiménez, Laureano & Guillén-Gosálbez, Gonzalo, 2018. "Productivity and energy efficiency assessment of existing industrial gases facilities via data envelopment analysis and the Malmquist index," Applied Energy, Elsevier, vol. 212(C), pages 1563-1577.
    19. Cheng, Gang & Zervopoulos, Panagiotis, 2012. "A proxy approach to dealing with the infeasibility problem in super-efficiency data envelopment analysis," MPRA Paper 42064, University Library of Munich, Germany.
    20. Vicente J. Bolós & Rafael Benítez & Vicente Coll-Serrano, 2023. "Continuous models combining slacks-based measures of efficiency and super-efficiency," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 31(2), pages 363-391, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:tefoso:v:111:y:2016:i:c:p:235-243. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.sciencedirect.com/science/journal/00401625 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.