IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v258y2017i2p607-616.html
   My bibliography  Save this article

Measuring systems sustainability with multi-criteria methods: A critical review

Author

Listed:
  • Diaz-Balteiro, L
  • González-Pachón, J.
  • Romero, C.

Abstract

Determining the sustainability of a system (e.g. through a criteria and indicators approach) has been the focus of research in many branches of science. Frequently, this research used multiple criteria decision making techniques. In this work, we analyze and critically assess the literature published on these topics. For this purpose, a set of 271 papers appearing in the ISI Web of Science database has been studied. The results show that these techniques have been applied to a great variety of problems, levels, and sectors, related to sustainability. Thus, up to 15 multiple criteria decision making techniques, which have been applied in 4 or more papers, have been identified. Those techniques have been grouped in 5 large clusters; the two most used being those called Analytic Hierarchical Process and Weighted Arithmetic Mean. On the other hand, in this work it has been verified that the use of multiple criteria decision making techniques hybridized with group decision-making techniques is quite common, and the use of both techniques for assessing sustainability problems has risen over the last few years. The aim of this hybridization process consists of including in the analysis the preferences of the stakeholders with respect to the indicators initially suggested. Finally, it has been seen that during the past few years there has been a great proliferation of works aggregating sustainability criteria by using this type of tool, which is undoubtedly a sign of the paramount importance of these techniques in this highly pluridisciplinary context.

Suggested Citation

  • Diaz-Balteiro, L & González-Pachón, J. & Romero, C., 2017. "Measuring systems sustainability with multi-criteria methods: A critical review," European Journal of Operational Research, Elsevier, vol. 258(2), pages 607-616.
  • Handle: RePEc:eee:ejores:v:258:y:2017:i:2:p:607-616
    DOI: 10.1016/j.ejor.2016.08.075
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221716307147
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ejor.2016.08.075?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Tarja Joro & Pekka Korhonen & Jyrki Wallenius, 1998. "Structural Comparison of Data Envelopment Analysis and Multiple Objective Linear Programming," Management Science, INFORMS, vol. 44(7), pages 962-970, July.
    2. Shmelev, Stanislav E. & Rodríguez-Labajos, Beatriz, 2009. "Corrigendum to "Dynamic multidimensional assessment of sustainability at the macro level: The case of Austria" [Ecological Economics 68 (2009) 2560-2573]," Ecological Economics, Elsevier, vol. 69(1), pages 209-209, November.
    3. Wang, Jiang-Jiang & Jing, You-Yin & Zhang, Chun-Fa & Zhao, Jun-Hong, 2009. "Review on multi-criteria decision analysis aid in sustainable energy decision-making," Renewable and Sustainable Energy Reviews, Elsevier, vol. 13(9), pages 2263-2278, December.
    4. Maxim, Alexandru, 2014. "Sustainability assessment of electricity generation technologies using weighted multi-criteria decision analysis," Energy Policy, Elsevier, vol. 65(C), pages 284-297.
    5. Adrianto, Luky & Matsuda, Yoshiaki & Sakuma, Yoshiaki, 2005. "Assessing local sustainability of fisheries system: a multi-criteria participatory approach with the case of Yoron Island, Kagoshima prefecture, Japan," Marine Policy, Elsevier, vol. 29(1), pages 9-23, January.
    6. Shmelev, Stanislav E. & Rodríguez-Labajos, Beatriz, 2009. "Dynamic multidimensional assessment of sustainability at the macro level: The case of Austria," Ecological Economics, Elsevier, vol. 68(10), pages 2560-2573, August.
    7. Caradonna, Jeremy L., 2014. "Sustainability: A History," OUP Catalogue, Oxford University Press, number 9780199372409, Decembrie.
    8. Michalopoulos, T. & Hogeveen, H. & Heuvelink, E. & Oude Lansink, A.G.J.M., 2013. "Public multi-criteria assessment for societal concerns and gradual labelling," Food Policy, Elsevier, vol. 40(C), pages 97-108.
    9. Jalilova, Gulnaz & Khadka, Chiranjeewee & Vacik, Harald, 2012. "Developing criteria and indicators for evaluating sustainable forest management: A case study in Kyrgyzstan," Forest Policy and Economics, Elsevier, vol. 21(C), pages 32-43.
    10. Özdemir, Enver Doruk & Härdtlein, Marlies & Jenssen, Till & Zech, Daniel & Eltrop, Ludger, 2011. "A confusion of tongues or the art of aggregating indicators--Reflections on four projective methodologies on sustainability measurement," Renewable and Sustainable Energy Reviews, Elsevier, vol. 15(5), pages 2385-2396, June.
    11. Lo Prete, Chiara & Hobbs, Benjamin F. & Norman, Catherine S. & Cano-Andrade, Sergio & Fuentes, Alejandro & von Spakovsky, Michael R. & Mili, Lamine, 2012. "Sustainability and reliability assessment of microgrids in a regional electricity market," Energy, Elsevier, vol. 41(1), pages 192-202.
    12. Elena Arce, María & Saavedra, Ángeles & Míguez, José L. & Granada, Enrique, 2015. "The use of grey-based methods in multi-criteria decision analysis for the evaluation of sustainable energy systems: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 47(C), pages 924-932.
    13. Troldborg, Mads & Heslop, Simon & Hough, Rupert L., 2014. "Assessing the sustainability of renewable energy technologies using multi-criteria analysis: Suitability of approach for national-scale assessments and associated uncertainties," Renewable and Sustainable Energy Reviews, Elsevier, vol. 39(C), pages 1173-1184.
    14. Charnes, A. & Cooper, W. W. & Rhodes, E., 1978. "Measuring the efficiency of decision making units," European Journal of Operational Research, Elsevier, vol. 2(6), pages 429-444, November.
    15. P. L. Yu, 1973. "A Class of Solutions for Group Decision Problems," Management Science, INFORMS, vol. 19(8), pages 936-946, April.
    16. Eric Neumayer, 2013. "Weak versus Strong Sustainability," Books, Edward Elgar Publishing, number 14993.
    17. Yoram Wind & Thomas L. Saaty, 1980. "Marketing Applications of the Analytic Hierarchy Process," Management Science, INFORMS, vol. 26(7), pages 641-658, July.
    18. Anthony Halog & Yosef Manik, 2011. "Advancing Integrated Systems Modelling Framework for Life Cycle Sustainability Assessment," Sustainability, MDPI, vol. 3(2), pages 1-31, February.
    19. Dylan Jones & Mehrdad Tamiz, 2010. "Practical Goal Programming," International Series in Operations Research and Management Science, Springer, edition 1, number 978-1-4419-5771-9, December.
    20. Hiltunen, Veikko & Kangas, Jyrki & Pykalainen, Jouni, 2008. "Voting methods in strategic forest planning -- Experiences from Metsahallitus," Forest Policy and Economics, Elsevier, vol. 10(3), pages 117-127, January.
    21. Erol, Ismail & Sencer, Safiye & Sari, Ramazan, 2011. "A new fuzzy multi-criteria framework for measuring sustainability performance of a supply chain," Ecological Economics, Elsevier, vol. 70(6), pages 1088-1100, April.
    22. Romero, Carlos, 2004. "A general structure of achievement function for a goal programming model," European Journal of Operational Research, Elsevier, vol. 153(3), pages 675-686, March.
    23. Blancas, Francisco J. & Caballero, Rafael & González, Mercedes & Lozano-Oyola, Macarena & Pérez, Fátima, 2010. "Goal programming synthetic indicators: An application for sustainable tourism in Andalusian coastal counties," Ecological Economics, Elsevier, vol. 69(11), pages 2158-2172, September.
    24. Shmelev, Stanislav E. & van den Bergh, Jeroen C.J.M., 2016. "Optimal diversity of renewable energy alternatives under multiple criteria: An application to the UK," Renewable and Sustainable Energy Reviews, Elsevier, vol. 60(C), pages 679-691.
    25. Herrera, F. & Herrera-Viedma, E. & Chiclana, F., 2001. "Multiperson decision-making based on multiplicative preference relations," European Journal of Operational Research, Elsevier, vol. 129(2), pages 372-385, March.
    26. Bernard Roy & Roman Slowinski, 2013. "Questions guiding the choice of a multicriteria decision aiding method," Post-Print hal-00874292, HAL.
    27. Dina Margrethe Aspen & Magnus Sparrevik & Annik Magerholm Fet, 2015. "Review of methods for sustainability appraisals in ship acquisition," Environment Systems and Decisions, Springer, vol. 35(3), pages 323-333, September.
    28. Greco, Salvatore & Matarazzo, Benedetto & Slowinski, Roman, 2001. "Rough sets theory for multicriteria decision analysis," European Journal of Operational Research, Elsevier, vol. 129(1), pages 1-47, February.
    29. Bertrand Mareschal & Jean Pierre Brans & Philippe Vincke, 1986. "How to select and how to rank projects: the Prométhée method," ULB Institutional Repository 2013/9307, ULB -- Universite Libre de Bruxelles.
    30. Denis Bouyssou & Thierry Marchant & Marc Pirlot & Alexis Tsoukiàs & Philippe Vincke, 2006. "Evaluation and Decision Models with Multiple Criteria," International Series in Operations Research and Management Science, Springer, number 978-0-387-31099-2, December.
    31. Brans, J. P. & Vincke, Ph. & Mareschal, B., 1986. "How to select and how to rank projects: The method," European Journal of Operational Research, Elsevier, vol. 24(2), pages 228-238, February.
    32. Giuseppe Munda & Michela Nardo & Michaela Saisana & Tanja Srebotnjak, 2009. "Measuring uncertainties in composite indicators of sustainability," International Journal of Environmental Technology and Management, Inderscience Enterprises Ltd, vol. 11(1/2/3), pages 7-26.
    33. Shmelev, S.E. & Powell, J.R., 2006. "Ecological-economic modelling for strategic regional waste management systems," Ecological Economics, Elsevier, vol. 59(1), pages 115-130, August.
    34. Miguel F. Salvado & Susana G. Azevedo & João C. O. Matias & Luís M. Ferreira, 2015. "Proposal of a Sustainability Index for the Automotive Industry," Sustainability, MDPI, vol. 7(2), pages 1-32, February.
    35. R. D. Banker & A. Charnes & W. W. Cooper, 1984. "Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis," Management Science, INFORMS, vol. 30(9), pages 1078-1092, September.
    36. Víctor Pérez & Alain Hernández & Flor Guerrero & María Amparo León & Christian Luiz Silva & Rafael Caballero, 2016. "Sustainability Ranking for Cuban Tourist Destinations Based on Composite Indexes," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 129(1), pages 425-444, October.
    37. Pohekar, S. D. & Ramachandran, M., 2004. "Application of multi-criteria decision making to sustainable energy planning--A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 8(4), pages 365-381, August.
    38. Gómez-Limón, José A. & Sanchez-Fernandez, Gabriela, 2010. "Empirical evaluation of agricultural sustainability using composite indicators," Ecological Economics, Elsevier, vol. 69(5), pages 1062-1075, March.
    39. Giuseppe Munda, 2008. "Social Multi-Criteria Evaluation for a Sustainable Economy," Springer Books, Springer, number 978-3-540-73703-2, November.
    40. Stefan Hajkowicz & Kerry Collins, 2007. "A Review of Multiple Criteria Analysis for Water Resource Planning and Management," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 21(9), pages 1553-1566, September.
    41. repec:hal:wpaper:hal-00874292 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Samira El Gibari & Trinidad Gómez & Francisco Ruiz, 2019. "Building composite indicators using multicriteria methods: a review," Journal of Business Economics, Springer, vol. 89(1), pages 1-24, February.
    2. Bortoluzzi, Mirian & Correia de Souza, Celso & Furlan, Marcelo, 2021. "Bibliometric analysis of renewable energy types using key performance indicators and multicriteria decision models," Renewable and Sustainable Energy Reviews, Elsevier, vol. 143(C).
    3. Ruiz, Francisco & El Gibari, Samira & Cabello, José M. & Gómez, Trinidad, 2020. "MRP-WSCI: Multiple reference point based weak and strong composite indicators," Omega, Elsevier, vol. 95(C).
    4. Urošević, Branka Gvozdenac & Marinović, Budimirka, 2021. "Ranking construction of small hydro power plants using multi-criteria decision analysis," Renewable Energy, Elsevier, vol. 172(C), pages 1174-1183.
    5. Ahmad, Salman & Nadeem, Abid & Akhanova, Gulzhanat & Houghton, Tom & Muhammad-Sukki, Firdaus, 2017. "Multi-criteria evaluation of renewable and nuclear resources for electricity generation in Kazakhstan," Energy, Elsevier, vol. 141(C), pages 1880-1891.
    6. Stanislav Edward Shmelev (ODID), "undated". "Multi-criteria Assessment of Ecosystems and Biodiversity: New Dimensions and Stakeholders in the South of France," QEH Working Papers qehwps181, Queen Elizabeth House, University of Oxford.
    7. Baudry, Gino & Macharis, Cathy & Vallée, Thomas, 2018. "Range-based Multi-Actor Multi-Criteria Analysis: A combined method of Multi-Actor Multi-Criteria Analysis and Monte Carlo simulation to support participatory decision making under uncertainty," European Journal of Operational Research, Elsevier, vol. 264(1), pages 257-269.
    8. Baumann, Manuel & Weil, Marcel & Peters, Jens F. & Chibeles-Martins, Nelson & Moniz, Antonio B., 2019. "A review of multi-criteria decision making approaches for evaluating energy storage systems for grid applications," Renewable and Sustainable Energy Reviews, Elsevier, vol. 107(C), pages 516-534.
    9. Daniel R. Georgiadis & Thomas A. Mazzuchi & Shahram Sarkani, 2013. "Using multi criteria decision making in analysis of alternatives for selection of enabling technology," Systems Engineering, John Wiley & Sons, vol. 16(3), pages 287-303, September.
    10. Amelia Bilbao-Terol & Mariano Jiménez & Mar Arenas-Parra, 2016. "A group decision making model based on goal programming with fuzzy hierarchy: an application to regional forest planning," Annals of Operations Research, Springer, vol. 245(1), pages 137-162, October.
    11. Bilgili, Faik & Zarali, Fulya & Ilgün, Miraç Fatih & Dumrul, Cüneyt & Dumrul, Yasemin, 2022. "The evaluation of renewable energy alternatives for sustainable development in Turkey using ‌intuitionistic‌ ‌fuzzy‌-TOPSIS method," Renewable Energy, Elsevier, vol. 189(C), pages 1443-1458.
    12. Alizadeh, Reza & Soltanisehat, Leili & Lund, Peter D. & Zamanisabzi, Hamed, 2020. "Improving renewable energy policy planning and decision-making through a hybrid MCDM method," Energy Policy, Elsevier, vol. 137(C).
    13. Strantzali, Eleni & Aravossis, Konstantinos & Livanos, Georgios A., 2017. "Evaluation of future sustainable electricity generation alternatives: The case of a Greek island," Renewable and Sustainable Energy Reviews, Elsevier, vol. 76(C), pages 775-787.
    14. Greco, Salvatore & Ishizaka, Alessio & Tasiou, Menelaos & Torrisi, Gianpiero, 2018. "σ-µ efficiency analysis: A new methodology for evaluating units through composite indices," MPRA Paper 83569, University Library of Munich, Germany.
    15. Roberto Cervelló Royo & Fernando García García & Francisco Guijarro-Martínez & Ismael Moya-Clemente, 2011. "Housing Ranking: a model of equilibrium between buyers and sellers expectations," ERSA conference papers ersa11p314, European Regional Science Association.
    16. Zhicheng Lai & Lei Li & Zhuomin Tao & Tao Li & Xiaoting Shi & Jialing Li & Xin Li, 2023. "Spatio-Temporal Evolution and Influencing Factors of Ecological Well-Being Performance from the Perspective of Strong Sustainability: A Case Study of the Three Gorges Reservoir Area, China," IJERPH, MDPI, vol. 20(3), pages 1-25, January.
    17. Stanislav Shmelev & Harrison Roger Brook, 2021. "Macro Sustainability across Countries: Key Sector Environmentally Extended Input-Output Analysis," Sustainability, MDPI, vol. 13(21), pages 1-46, October.
    18. Greco, Salvatore & Ishizaka, Alessio & Tasiou, Menelaos & Torrisi, Gianpiero, 2019. "Sigma-Mu efficiency analysis: A methodology for evaluating units through composite indicators," European Journal of Operational Research, Elsevier, vol. 278(3), pages 942-960.
    19. Rahimdel, Mohammad Javad & Noferesti, Hossein, 2020. "Investment preferences of Iran's mineral extraction sector with a focus on the productivity of the energy consumption, water and labor force," Resources Policy, Elsevier, vol. 67(C).
    20. Hassan, Mohammad Nurul & Hawas, Yaser E. & Ahmed, Kamran, 2013. "A multi-dimensional framework for evaluating the transit service performance," Transportation Research Part A: Policy and Practice, Elsevier, vol. 50(C), pages 47-61.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:258:y:2017:i:2:p:607-616. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.