IDEAS home Printed from https://ideas.repec.org/a/wsi/ijitdm/v15y2016i01ns0219622015500315.html
   My bibliography  Save this article

Simulation-Based Evaluation of Criteria Rank-Weighting Methods in Multi-Criteria Decision-Making

Author

Listed:
  • Hesham K. Alfares

    (Systems Engineering Department, King Fahd University of Petroleum & Minerals, Dhahran 31261, Saudi Arabia)

  • Salih O. Duffuaa

    (Systems Engineering Department, King Fahd University of Petroleum & Minerals, Dhahran 31261, Saudi Arabia)

Abstract

This paper presents a simulation study to assess the performance of the five known methods for converting ranks of several criteria into weights in multi-criteria decision-making. The five methods assessed for converting criteria ranks into weights are: rank- sum (RS) weights, rank reciprocal (RR) weights, rank order centroid (ROC) weights, geometric weights (GW), and variable-slope linear (VSL) weights. The methods are compared in terms of weight estimation accuracy considering different numbers of criteria and decision makers’ (MS) preference structures. Alternative preference structures are represented by different probability distributions of randomly generated criteria weights, namely the uniform, normal, and exponential distributions. Results of the simulation experiments indicate that no single method is consistently superior to all others. On average, RS is best for uniform weights, VSL is best for normal weights, and ROC is best for exponential weights. However, for any multi-criteria decision-making (MCDM) problem, the best method for converting criteria ranks into weights depends on both the number of criteria and the weight distribution.

Suggested Citation

  • Hesham K. Alfares & Salih O. Duffuaa, 2016. "Simulation-Based Evaluation of Criteria Rank-Weighting Methods in Multi-Criteria Decision-Making," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 15(01), pages 43-61, January.
    • Handle: RePEc:wsi:ijitdm:v:15:y:2016:i:01:n:s0219622015500315
    DOI: 10.1142/S0219622015500315
    as

    Download full text from publisher

    File URL: http://www.worldscientific.com/doi/abs/10.1142/S0219622015500315
    Download Restriction: Access to full text is restricted to subscribers
    File URL: https://libkey.io/10.1142/S0219622015500315?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Peng, Yi & Kou, Gang & Wang, Guoxun & Shi, Yong, 2011. "FAMCDM: A fusion approach of MCDM methods to rank multiclass classification algorithms," Omega, Elsevier, vol. 39(6), pages 677-689, December.
    2. Lahdelma, Risto & Miettinen, Kaisa & Salminen, Pekka, 2003. "Ordinal criteria in stochastic multicriteria acceptability analysis (SMAA)," European Journal of Operational Research, Elsevier, vol. 147(1), pages 117-127, May.
    3. James L. Corner & Craig W. Kirkwood, 1991. "Decision Analysis Applications in the Operations Research Literature, 1970–1989," Operations Research, INFORMS, vol. 39(2), pages 206-219, April.
    4. Katrin Borcherding & Thomas Eppel & Detlof von Winterfeldt, 1991. "Comparison of Weighting Judgments in Multiattribute Utility Measurement," Management Science, INFORMS, vol. 37(12), pages 1603-1619, December.
    5. Van Ittersum, Koert & Pennings, Joost M.E. & Wansink, Brian & van Trijp, Hans C.M., 2007. "The validity of attribute-importance measurement: A review," Journal of Business Research, Elsevier, vol. 60(11), pages 1177-1190, November.
    6. Gabriel, Steven A. & Kumar, Satheesh & Ordonez, Javier & Nasserian, Amirali, 2006. "A multiobjective optimization model for project selection with probabilistic considerations," Socio-Economic Planning Sciences, Elsevier, vol. 40(4), pages 297-313, December.
    7. Bottomley, Paul A. & Doyle, John R., 2001. "A comparison of three weight elicitation methods: good, better, and best," Omega, Elsevier, vol. 29(6), pages 553-560, December.
    8. Gang Kou & Yanqun Lu & Yi Peng & Yong Shi, 2012. "Evaluation Of Classification Algorithms Using Mcdm And Rank Correlation," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 11(01), pages 197-225.
    9. Ho, William & Xu, Xiaowei & Dey, Prasanta K., 2010. "Multi-criteria decision making approaches for supplier evaluation and selection: A literature review," European Journal of Operational Research, Elsevier, vol. 202(1), pages 16-24, April.
    10. Ahn, Byeong Seok, 2011. "Compatible weighting method with rank order centroid: Maximum entropy ordered weighted averaging approach," European Journal of Operational Research, Elsevier, vol. 212(3), pages 552-559, August.
    11. Hesham K. Alfares & Salih O. Duffuaa, 2008. "Determining Aggregate Criteria Weights From Criteria Rankings By A Group Of Decision Makers," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 7(04), pages 769-781.
    12. Kou, Gang & Lin, Changsheng, 2014. "A cosine maximization method for the priority vector derivation in AHP," European Journal of Operational Research, Elsevier, vol. 235(1), pages 225-232.
    13. Henao, Felipe & Cherni, Judith A. & Jaramillo, Patricia & Dyner, Isaac, 2012. "A multicriteria approach to sustainable energy supply for the rural poor," European Journal of Operational Research, Elsevier, vol. 218(3), pages 801-809.
    14. Ergu, Daji & Kou, Gang & Peng, Yi & Shi, Yong, 2011. "A simple method to improve the consistency ratio of the pair-wise comparison matrix in ANP," European Journal of Operational Research, Elsevier, vol. 213(1), pages 246-259, August.
    15. Solymosi, Tamas & Dombi, Jozsef, 1986. "A method for determining the weights of criteria: The centralized weights," European Journal of Operational Research, Elsevier, vol. 26(1), pages 35-41, July.
    16. A Jessop, 2011. "Using imprecise estimates for weights," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 62(6), pages 1048-1055, June.
    17. F. Hutton Barron & Bruce E. Barrett, 1996. "Decision Quality Using Ranked Attribute Weights," Management Science, INFORMS, vol. 42(11), pages 1515-1523, November.
    18. Moshkovich, Helen M. & Mechitov, Alexander I. & Olson, David L., 2002. "Ordinal judgments in multiattribute decision analysis," European Journal of Operational Research, Elsevier, vol. 137(3), pages 625-641, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Renan Felinto de Farias Aires & Camila Cristina Rodrigues Salgado, 2022. "A Multi-Criteria Approach to Assess the Performance of the Brazilian Unified Health System," IJERPH, MDPI, vol. 19(18), pages 1-13, September.
    2. Mohamad Shahiir Saidin & Lai Soon Lee & Siti Mahani Marjugi & Muhammad Zaini Ahmad & Hsin-Vonn Seow, 2023. "Fuzzy Method Based on the Removal Effects of Criteria (MEREC) for Determining Objective Weights in Multi-Criteria Decision-Making Problems," Mathematics, MDPI, vol. 11(6), pages 1-20, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Aron Larsson & Mona Riabacke & Mats Danielson & Love Ekenberg, 2015. "Cardinal and Rank Ordering of Criteria — Addressing Prescription within Weight Elicitation," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 14(06), pages 1299-1330, November.
    2. Roger Chapman Burk & Richard M. Nehring, 2023. "An Empirical Comparison of Rank-Based Surrogate Weights in Additive Multiattribute Decision Analysis," Decision Analysis, INFORMS, vol. 20(1), pages 55-72, March.
    3. de Almeida Filho, Adiel T. & Clemente, Thárcylla R.N. & Morais, Danielle Costa & de Almeida, Adiel Teixeira, 2018. "Preference modeling experiments with surrogate weighting procedures for the PROMETHEE method," European Journal of Operational Research, Elsevier, vol. 264(2), pages 453-461.
    4. Jessop, Alan, 2014. "IMP: A decision aid for multiattribute evaluation using imprecise weight estimates," Omega, Elsevier, vol. 49(C), pages 18-29.
    5. Kun Chen & Gang Kou & J. Michael Tarn & Yan Song, 2015. "Bridging the gap between missing and inconsistent values in eliciting preference from pairwise comparison matrices," Annals of Operations Research, Springer, vol. 235(1), pages 155-175, December.
    6. Eleonora Bottani & Piera Centobelli & Teresa Murino & Ehsan Shekarian, 2018. "A QFD-ANP Method for Supplier Selection with Benefits, Opportunities, Costs and Risks Considerations," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 17(03), pages 911-939, May.
    7. Stephen P. Chambal & Jeffery D. Weir & Yucel R. Kahraman & Alex J. Gutman, 2011. "A Practical Procedure for Customizable One-Way Sensitivity Analysis in Additive Value Models," Decision Analysis, INFORMS, vol. 8(4), pages 303-321, December.
    8. Raquel González del Pozo & Luis C. Dias & José Luis García-Lapresta, 2020. "Using Different Qualitative Scales in a Multi-Criteria Decision-Making Procedure," Mathematics, MDPI, vol. 8(3), pages 1-20, March.
    9. Peide Liu & Peng Wang, 2017. "Some Improved Linguistic Intuitionistic Fuzzy Aggregation Operators and Their Applications to Multiple-Attribute Decision Making," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 16(03), pages 817-850, May.
    10. Hsu-Shih Shih, 2016. "A Mixed-Data Evaluation in Group TOPSIS with Differentiated Decision Power," Group Decision and Negotiation, Springer, vol. 25(3), pages 537-565, May.
    11. Lienert, Judit & Duygan, Mert & Zheng, Jun, 2016. "Preference stability over time with multiple elicitation methods to support wastewater infrastructure decision-making," European Journal of Operational Research, Elsevier, vol. 253(3), pages 746-760.
    12. Thomas L. Saaty & Daji Ergu, 2015. "When is a Decision-Making Method Trustworthy? Criteria for Evaluating Multi-Criteria Decision-Making Methods," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 14(06), pages 1171-1187, November.
    13. Ewa Roszkowska, 2020. "The extention rank ordering criteria weighting methods in fuzzy enviroment," Operations Research and Decisions, Wroclaw University of Science and Technology, Faculty of Management, vol. 30(2), pages 91-114.
    14. Peide Liu & Lili Zhang & Xi Liu & Peng Wang, 2016. "Multi-Valued Neutrosophic Number Bonferroni Mean Operators with their Applications in Multiple Attribute Group Decision Making," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 15(05), pages 1181-1210, September.
    15. Kou, Gang & Lin, Changsheng, 2014. "A cosine maximization method for the priority vector derivation in AHP," European Journal of Operational Research, Elsevier, vol. 235(1), pages 225-232.
    16. Fei Teng & Peide Liu & Li Zhang & Juan Zhao, 2019. "Multiple Attribute Decision-Making Methods with Unbalanced Linguistic Variables Based on Maclaurin Symmetric Mean Operators," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 18(01), pages 105-146, January.
    17. de Almeida, Jonatas Araujo & Costa, Ana Paula Cabral Seixas & de Almeida-Filho, Adiel Teixeira, 2016. "A new method for elicitation of criteria weights in additive models: Flexible and interactive tradeoffAuthor-Name: de Almeida, Adiel Teixeira," European Journal of Operational Research, Elsevier, vol. 250(1), pages 179-191.
    18. Zhang, Huanhuan & Kou, Gang & Peng, Yi, 2019. "Soft consensus cost models for group decision making and economic interpretations," European Journal of Operational Research, Elsevier, vol. 277(3), pages 964-980.
    19. Ayfer Basar & Özgür Kabak & Y. Ilker Topcu, 2017. "A Decision Support Methodology for Locating Bank Branches: A Case Study in Turkey," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 16(01), pages 59-86, January.
    20. Edmundas Kazimieras Zavadskas & Valentinas Podvezko, 2016. "Integrated Determination of Objective Criteria Weights in MCDM," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 15(02), pages 267-283, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wsi:ijitdm:v:15:y:2016:i:01:n:s0219622015500315. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Tai Tone Lim (email available below). General contact details of provider: http://www.worldscinet.com/ijitdm/ijitdm.shtml .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.