IDEAS home Printed from https://ideas.repec.org/a/wsi/ijitdm/v17y2018i04ns0219622018500177.html
   My bibliography  Save this article

Characterization of the Effectiveness of Several Outranking-Based Multi-Criteria Sorting Methods

Author

Listed:
  • Eduardo Fernandez

    (Autonomous University of Sinaloa, Blvd. Las Americas s/n, Ciudad Universitaria, Culiacan, Sinaloa, Mexico 80040, Mexico)

  • Jorge Navarro

    (Autonomous University of Sinaloa, Blvd. Las Americas s/n, Ciudad Universitaria, Culiacan, Sinaloa, Mexico 80040, Mexico)

  • Rafael Olmedo

    (Autonomous University of Sinaloa, Blvd. Las Americas s/n, Ciudad Universitaria, Culiacan, Sinaloa, Mexico 80040, Mexico)

Abstract

ELECTRE TRI-C, ELECTRE TRI-nC and THESEUS are recently proposed outranking-based multi-criteria sorting methods. The effectiveness of each method is here characterized by (i) its capacity for suggesting precise and appropriate assignments, (ii) the probability of suggesting imprecise assignments, and (iii) the probability of suggesting wrong assignments. We are also interested in studying to what extent the effectiveness of the methods is influenced by the number of criteria and categories, the cardinality of the reference set (in case of ELECTRE TRI-nC and THESEUS), and the way in which this set is chosen or populated. Our conclusions are supported by extensive computer experiments. The results underline the importance of (a) the number of reference actions and their distribution per category, (b) the number of criteria, and (c) the way in which the reference set is populated. These results help to identify contexts for which a particular method may be more appropriate.

Suggested Citation

  • Eduardo Fernandez & Jorge Navarro & Rafael Olmedo, 2018. "Characterization of the Effectiveness of Several Outranking-Based Multi-Criteria Sorting Methods," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 17(04), pages 1047-1084, July.
    • Handle: RePEc:wsi:ijitdm:v:17:y:2018:i:04:n:s0219622018500177
    DOI: 10.1142/S0219622018500177
    as

    Download full text from publisher

    File URL: http://www.worldscientific.com/doi/abs/10.1142/S0219622018500177
    Download Restriction: Access to full text is restricted to subscribers
    File URL: https://libkey.io/10.1142/S0219622018500177?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Peng, Yi & Kou, Gang & Wang, Guoxun & Shi, Yong, 2011. "FAMCDM: A fusion approach of MCDM methods to rank multiclass classification algorithms," Omega, Elsevier, vol. 39(6), pages 677-689, December.
    2. Fernandez, Eduardo & Navarro, Jorge, 2011. "A new approach to multi-criteria sorting based on fuzzy outranking relations: The THESEUS method," European Journal of Operational Research, Elsevier, vol. 213(2), pages 405-413, September.
    3. Bernard Roy & Roman Slowinski, 2013. "Questions guiding the choice of a multicriteria decision aiding method," Post-Print hal-00874292, HAL.
    4. Greco, Salvatore & Mousseau, Vincent & Slowinski, Roman, 2010. "Multiple criteria sorting with a set of additive value functions," European Journal of Operational Research, Elsevier, vol. 207(3), pages 1455-1470, December.
    5. Araz, Ceyhun & Ozkarahan, Irem, 2007. "Supplier evaluation and management system for strategic sourcing based on a new multicriteria sorting procedure," International Journal of Production Economics, Elsevier, vol. 106(2), pages 585-606, April.
    6. Greco, Salvatore & Matarazzo, Benedetto & Slowinski, Roman, 2001. "Rough sets theory for multicriteria decision analysis," European Journal of Operational Research, Elsevier, vol. 129(1), pages 1-47, February.
    7. Almeida-Dias, J. & Figueira, J.R. & Roy, B., 2010. "Electre Tri-C: A multiple criteria sorting method based on characteristic reference actions," European Journal of Operational Research, Elsevier, vol. 204(3), pages 565-580, August.
    8. Pinto, F.S. & Costa, A.S. & Figueira, J.R. & Marques, R.C., 2017. "The quality of service: An overall performance assessment for water utilities," Omega, Elsevier, vol. 69(C), pages 115-125.
    9. Salminen, Pekka & Hokkanen, Joonas & Lahdelma, Risto, 1998. "Comparing multicriteria methods in the context of environmental problems," European Journal of Operational Research, Elsevier, vol. 104(3), pages 485-496, February.
    10. Thomas L. Saaty & Daji Ergu, 2015. "When is a Decision-Making Method Trustworthy? Criteria for Evaluating Multi-Criteria Decision-Making Methods," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 14(06), pages 1171-1187, November.
    11. Doumpos, M. & Kosmidou, K. & Baourakis, G. & Zopounidis, C., 2002. "Credit risk assessment using a multicriteria hierarchical discrimination approach: A comparative analysis," European Journal of Operational Research, Elsevier, vol. 138(2), pages 392-412, April.
    12. Dembczynski, Krzysztof & Greco, Salvatore & Slowinski, Roman, 2009. "Rough set approach to multiple criteria classification with imprecise evaluations and assignments," European Journal of Operational Research, Elsevier, vol. 198(2), pages 626-636, October.
    13. Doumpos, M. & Marinakis, Y. & Marinaki, M. & Zopounidis, C., 2009. "An evolutionary approach to construction of outranking models for multicriteria classification: The case of the ELECTRE TRI method," European Journal of Operational Research, Elsevier, vol. 199(2), pages 496-505, December.
    14. Opricovic, Serafim & Tzeng, Gwo-Hshiung, 2004. "Compromise solution by MCDM methods: A comparative analysis of VIKOR and TOPSIS," European Journal of Operational Research, Elsevier, vol. 156(2), pages 445-455, July.
    15. Koksalan, Murat & Ulu, Canan, 2003. "An interactive approach for placing alternatives in preference classes," European Journal of Operational Research, Elsevier, vol. 144(2), pages 429-439, January.
    16. Mousseau, Vincent & Dias, Luis, 2004. "Valued outranking relations in ELECTRE providing manageable disaggregation procedures," European Journal of Operational Research, Elsevier, vol. 156(2), pages 467-482, July.
    17. Murat Köksalan & Vincent Mousseau & Özgür Özpeynirci & Selin Bilgin Özpeynirci, 2009. "A new outranking‐based approach for assigning alternatives to ordered classes," Naval Research Logistics (NRL), John Wiley & Sons, vol. 56(1), pages 74-85, February.
    18. Doumpos, Michael & Zopounidis, Constantin, 2004. "Developing sorting models using preference disaggregation analysis: An experimental investigation," European Journal of Operational Research, Elsevier, vol. 154(3), pages 585-598, May.
    19. Gang Kou & Yanqun Lu & Yi Peng & Yong Shi, 2012. "Evaluation Of Classification Algorithms Using Mcdm And Rank Correlation," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 11(01), pages 197-225.
    20. Doumpos, Michael & Zopounidis, Constantin, 2004. "A multicriteria classification approach based on pairwise comparisons," European Journal of Operational Research, Elsevier, vol. 158(2), pages 378-389, October.
    21. Patrice Perny, 1998. "Multicriteria filtering methods based onconcordance and non-discordance principles," Annals of Operations Research, Springer, vol. 80(0), pages 137-165, January.
    22. Almeida-Dias, J. & Figueira, J.R. & Roy, B., 2012. "A multiple criteria sorting method where each category is characterized by several reference actions: The Electre Tri-nC method," European Journal of Operational Research, Elsevier, vol. 217(3), pages 567-579.
    23. Tervonen, Tommi & Figueira, José Rui & Lahdelma, Risto & Dias, Juscelino Almeida & Salminen, Pekka, 2009. "A stochastic method for robustness analysis in sorting problems," European Journal of Operational Research, Elsevier, vol. 192(1), pages 236-242, January.
    24. Bouyssou, Denis & Marchant, Thierry, 2015. "On the relations between ELECTRE TRI-B and ELECTRE TRI-C and on a new variant of ELECTRE TRI-B," European Journal of Operational Research, Elsevier, vol. 242(1), pages 201-211.
    25. Hajkowicz, Stefan & Higgins, Andrew, 2008. "A comparison of multiple criteria analysis techniques for water resource management," European Journal of Operational Research, Elsevier, vol. 184(1), pages 255-265, January.
    26. Kou, Gang & Ergu, Daji & Shang, Jennifer, 2014. "Enhancing data consistency in decision matrix: Adapting Hadamard model to mitigate judgment contradiction," European Journal of Operational Research, Elsevier, vol. 236(1), pages 261-271.
    27. Brito, Anderson J. & de Almeida, Adiel Teixeira & Mota, Caroline M.M., 2010. "A multicriteria model for risk sorting of natural gas pipelines based on ELECTRE TRI integrating Utility Theory," European Journal of Operational Research, Elsevier, vol. 200(3), pages 812-821, February.
    28. Kosmidou K. & Doumpos M. & Zopounidis C., 2002. "A Multicriteria Hierarchical Discrimination Approach for Credit Risk Problems," European Research Studies Journal, European Research Studies Journal, vol. 0(1-2), pages 53-68, January -.
    29. Govindan, Kannan & Jepsen, Martin Brandt, 2016. "ELECTRE: A comprehensive literature review on methodologies and applications," European Journal of Operational Research, Elsevier, vol. 250(1), pages 1-29.
    30. Eduardo Fernandez & Jorge Navarro & Edgar Covantes & Jorge Rodriguez, 2017. "Analysis of the effectiveness of the theseus multi-criteria sorting method: theoretical remarks and experimental evidence," TOP: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 25(2), pages 314-339, July.
    31. Fortemps, Philippe & Greco, Salvatore & Slowinski, Roman, 2008. "Multicriteria decision support using rules that represent rough-graded preference relations," European Journal of Operational Research, Elsevier, vol. 188(1), pages 206-223, July.
    32. Olson, David L., 2001. "Comparison of three multicriteria methods to predict known outcomes," European Journal of Operational Research, Elsevier, vol. 130(3), pages 576-587, May.
    33. Zanakis, Stelios H. & Solomon, Anthony & Wishart, Nicole & Dublish, Sandipa, 1998. "Multi-attribute decision making: A simulation comparison of select methods," European Journal of Operational Research, Elsevier, vol. 107(3), pages 507-529, June.
    34. Zamani-Sabzi, Hamed & King, James Phillip & Gard, Charlotte C. & Abudu, Shalamu, 2016. "Statistical and analytical comparison of multi-criteria decision-making techniques under fuzzy environment," Operations Research Perspectives, Elsevier, vol. 3(C), pages 92-117.
    35. Bouyssou, Denis & Marchant, Thierry, 2015. "On the relations between ELECTRE TRI-B and ELECTRE TRI-C and on a new variant of ELECTRE TRI-B," European Journal of Operational Research, Elsevier, vol. 242(1), pages 201-211.
    36. Fernandez, Eduardo & Navarro, Jorge & Duarte, Alfonso, 2008. "Multicriteria sorting using a valued preference closeness relation," European Journal of Operational Research, Elsevier, vol. 185(2), pages 673-686, March.
    37. Ishizaka, Alessio & Nemery, Philippe, 2014. "Assigning machines to incomparable maintenance strategies with ELECTRE-SORT," Omega, Elsevier, vol. 47(C), pages 45-59.
    38. Ishizaka, Alessio & Siraj, Sajid, 2018. "Are multi-criteria decision-making tools useful? An experimental comparative study of three methods," European Journal of Operational Research, Elsevier, vol. 264(2), pages 462-471.
    39. Butler, John & Jia, Jianmin & Dyer, James, 1997. "Simulation techniques for the sensitivity analysis of multi-criteria decision models," European Journal of Operational Research, Elsevier, vol. 103(3), pages 531-546, December.
    40. Guitouni, Adel & Martel, Jean-Marc, 1998. "Tentative guidelines to help choosing an appropriate MCDA method," European Journal of Operational Research, Elsevier, vol. 109(2), pages 501-521, September.
    41. repec:hal:wpaper:hal-00874292 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mladen Krstić & Giulio Paolo Agnusdei & Snežana Tadić & Milovan Kovač & Pier Paolo Miglietta, 2023. "A Novel Axial-Distance-Based Aggregated Measurement (ADAM) Method for the Evaluation of Agri-Food Circular-Economy-Based Business Models," Mathematics, MDPI, vol. 11(6), pages 1-27, March.
    2. Fernández, Eduardo & Figueira, José Rui & Navarro, Jorge, 2020. "Interval-based extensions of two outranking methods for multi-criteria ordinal classification," Omega, Elsevier, vol. 95(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Fernández, Eduardo & Figueira, José Rui & Navarro, Jorge & Solares, Efrain, 2023. "A generalized approach to ordinal classification based on the comparison of actions with either limiting or characteristic profiles," European Journal of Operational Research, Elsevier, vol. 305(3), pages 1309-1322.
    2. Fernandez, Eduardo & Navarro, Jorge, 2011. "A new approach to multi-criteria sorting based on fuzzy outranking relations: The THESEUS method," European Journal of Operational Research, Elsevier, vol. 213(2), pages 405-413, September.
    3. Eduardo Fernández & José Rui Figueira & Jorge Navarro, 2023. "A theoretical look at ordinal classification methods based on comparing actions with limiting boundaries between adjacent classes," Annals of Operations Research, Springer, vol. 325(2), pages 819-843, June.
    4. Govindan, Kannan & Jepsen, Martin Brandt, 2016. "ELECTRE: A comprehensive literature review on methodologies and applications," European Journal of Operational Research, Elsevier, vol. 250(1), pages 1-29.
    5. Cinelli, Marco & Kadziński, Miłosz & Miebs, Grzegorz & Gonzalez, Michael & Słowiński, Roman, 2022. "Recommending multiple criteria decision analysis methods with a new taxonomy-based decision support system," European Journal of Operational Research, Elsevier, vol. 302(2), pages 633-651.
    6. Khaled Belahcène & Vincent Mousseau & Wassila Ouerdane & Marc Pirlot & Olivier Sobrie, 2023. "Multiple criteria sorting models and methods—Part I: survey of the literature," 4OR, Springer, vol. 21(1), pages 1-46, March.
    7. Fu-Ling Cai & Xiuwu Liao & Kan-Liang Wang, 2012. "An interactive sorting approach based on the assignment examples of multiple decision makers with different priorities," Annals of Operations Research, Springer, vol. 197(1), pages 87-108, August.
    8. Eduardo Fernandez & Jorge Navarro & Edgar Covantes & Jorge Rodriguez, 2017. "Analysis of the effectiveness of the theseus multi-criteria sorting method: theoretical remarks and experimental evidence," TOP: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 25(2), pages 314-339, July.
    9. Thomas L. Saaty & Daji Ergu, 2015. "When is a Decision-Making Method Trustworthy? Criteria for Evaluating Multi-Criteria Decision-Making Methods," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 14(06), pages 1171-1187, November.
    10. Jindong Qin & Yingying Liang & Luis Martinez & Alessio Ishizaka & Witold Pedrycz, 2023. "ORESTE-SORT: a novel multiple criteria sorting method for sorting port group competitiveness," Annals of Operations Research, Springer, vol. 325(2), pages 875-909, June.
    11. Eduardo Fernandez & Jorge Navarro & Efrain Solares, 2021. "A theoretical look at ordinal classification methods based on reference sets composed of characteristic actions," Papers 2107.04656, arXiv.org.
    12. Denis Bouyssou & Thierry Marchant & Marc Pirlot, 2023. "A theoretical look at Electre Tri-nB and related sorting models," 4OR, Springer, vol. 21(1), pages 1-31, March.
    13. Almeida-Dias, J. & Figueira, J.R. & Roy, B., 2012. "A multiple criteria sorting method where each category is characterized by several reference actions: The Electre Tri-nC method," European Journal of Operational Research, Elsevier, vol. 217(3), pages 567-579.
    14. Murat Köksalan & Vincent Mousseau & Selin Özpeynirci, 2017. "Multi-Criteria Sorting with Category Size Restrictions," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 16(01), pages 5-23, January.
    15. Fernández, Eduardo & Figueira, José Rui & Navarro, Jorge, 2020. "Interval-based extensions of two outranking methods for multi-criteria ordinal classification," Omega, Elsevier, vol. 95(C).
    16. Liu, Jiapeng & Liao, Xiuwu & Yang, Jian-bo, 2015. "A group decision-making approach based on evidential reasoning for multiple criteria sorting problem with uncertainty," European Journal of Operational Research, Elsevier, vol. 246(3), pages 858-873.
    17. Tlili, Ali & Belahcène, Khaled & Khaled, Oumaima & Mousseau, Vincent & Ouerdane, Wassila, 2022. "Learning non-compensatory sorting models using efficient SAT/MaxSAT formulations," European Journal of Operational Research, Elsevier, vol. 298(3), pages 979-1006.
    18. Francesco Ciardiello & Andrea Genovese, 2023. "A comparison between TOPSIS and SAW methods," Annals of Operations Research, Springer, vol. 325(2), pages 967-994, June.
    19. Fernandez, Eduardo & Navarro, Jorge & Bernal, Sergio, 2009. "Multicriteria sorting using a valued indifference relation under a preference disaggregation paradigm," European Journal of Operational Research, Elsevier, vol. 198(2), pages 602-609, October.
    20. Hatami-Marbini, Adel & Tavana, Madjid, 2011. "An extension of the Electre I method for group decision-making under a fuzzy environment," Omega, Elsevier, vol. 39(4), pages 373-386, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wsi:ijitdm:v:17:y:2018:i:04:n:s0219622018500177. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Tai Tone Lim (email available below). General contact details of provider: http://www.worldscinet.com/ijitdm/ijitdm.shtml .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.