IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v8y2020i12p2200-d459980.html
   My bibliography  Save this article

The Influence of Criteria Selection Method on Consistency of Pairwise Comparison

Author

Listed:
  • Vladimír Bureš

    (Faculty of Informatics and Management, University of Hradec Kralove, Rokitanskeho 62, 50003 Hradec Kralove, Czech Republic
    These authors contributed equally to this work.)

  • Jiří Cabal

    (Faculty of Informatics and Management, University of Hradec Kralove, Rokitanskeho 62, 50003 Hradec Kralove, Czech Republic
    These authors contributed equally to this work.)

  • Pavel Čech

    (Faculty of Informatics and Management, University of Hradec Kralove, Rokitanskeho 62, 50003 Hradec Kralove, Czech Republic
    These authors contributed equally to this work.)

  • Karel Mls

    (Faculty of Informatics and Management, University of Hradec Kralove, Rokitanskeho 62, 50003 Hradec Kralove, Czech Republic
    These authors contributed equally to this work.)

  • Daniela Ponce

    (Faculty of Informatics and Management, University of Hradec Kralove, Rokitanskeho 62, 50003 Hradec Kralove, Czech Republic
    These authors contributed equally to this work.)

Abstract

The more criteria a human decision involves, the more inconsistent the decision. This study experimentally examines the effect on the degree of pairwise comparison inconsistency by using the (im)possibility of selecting the criteria for the evaluation and the size of the decision-making problem. A total of 358 participants completed objective and subjective tasks. While the former was associated with one possible correct solution, there was no single correct solution for the latter. The design of the experiment enabled the acquisition of eight groups in which the degree of inconsistency was quantified using three inconsistency indices (the Consistency Index, the Consistency Ratio and the Euclidean distance) and these were analysed by the repeated measures ANOVA. The results show a significant dependence of the degree of inconsistency on the method of determining the criteria for pairwise evaluation. If participants are randomly given the criteria, then with more criteria, the overall inconsistency of the comparison decreases. If the participants can themselves choose the criteria for the comparison, then with more criteria, the overall inconsistency of the comparison increases. This statistical dependence exists only for males. For females, the dependence is the opposite, but it is not statistically significant.

Suggested Citation

  • Vladimír Bureš & Jiří Cabal & Pavel Čech & Karel Mls & Daniela Ponce, 2020. "The Influence of Criteria Selection Method on Consistency of Pairwise Comparison," Mathematics, MDPI, vol. 8(12), pages 1-13, December.
    • Handle: RePEc:gam:jmathe:v:8:y:2020:i:12:p:2200-:d:459980
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/8/12/2200/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/8/12/2200/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Aguaron, Juan & Moreno-Jimenez, Jose Maria, 2003. "The geometric consistency index: Approximated thresholds," European Journal of Operational Research, Elsevier, vol. 147(1), pages 137-145, May.
    2. Sándor Bozóki & Linda Dezső & Attila Poesz & József Temesi, 2013. "Analysis of pairwise comparison matrices: an empirical research," Annals of Operations Research, Springer, vol. 211(1), pages 511-528, December.
    3. Matteo Brunelli & Luisa Canal & Michele Fedrizzi, 2013. "Inconsistency indices for pairwise comparison matrices: a numerical study," Annals of Operations Research, Springer, vol. 211(1), pages 493-509, December.
    4. Charness, Gary & Gneezy, Uri & Kuhn, Michael A., 2012. "Experimental methods: Between-subject and within-subject design," Journal of Economic Behavior & Organization, Elsevier, vol. 81(1), pages 1-8.
    5. Aguarón, Juan & Escobar, María Teresa & Moreno-Jiménez, José María, 2021. "Reducing inconsistency measured by the geometric consistency index in the analytic hierarchy process," European Journal of Operational Research, Elsevier, vol. 288(2), pages 576-583.
    6. Schreiber, Philipp & Weber, Martin, 2016. "Time inconsistent preferences and the annuitization decision," Journal of Economic Behavior & Organization, Elsevier, vol. 129(C), pages 37-55.
    7. Lin, Chang-Chun, 2007. "A revised framework for deriving preference values from pairwise comparison matrices," European Journal of Operational Research, Elsevier, vol. 176(2), pages 1145-1150, January.
    8. Bice Cavallo & Alessio Ishizaka & Maria Grazia Olivieri & Massimo Squillante, 2019. "Comparing inconsistency of pairwise comparison matrices depending on entries," Journal of the Operational Research Society, Taylor & Francis Journals, vol. 70(5), pages 842-850, May.
    9. Liang, Fuqi & Brunelli, Matteo & Rezaei, Jafar, 2020. "Consistency issues in the best worst method: Measurements and thresholds," Omega, Elsevier, vol. 96(C).
    10. Sándor Bozóki & János Fülöp & Attila Poesz, 2015. "On reducing inconsistency of pairwise comparison matrices below an acceptance threshold," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 23(4), pages 849-866, December.
    11. Changsheng Lin & Gang Kou & Daji Ergu, 2013. "An improved statistical approach for consistency test in AHP," Annals of Operations Research, Springer, vol. 211(1), pages 289-299, December.
    12. Ralph L. Keeney, 2002. "Common Mistakes in Making Value Trade-Offs," Operations Research, INFORMS, vol. 50(6), pages 935-945, December.
    13. Wade D. Cook & Boaz Golany & Moshe Kress & Michal Penn & Tal Raviv, 2005. "Optimal Allocation of Proposals to Reviewers to Facilitate Effective Ranking," Management Science, INFORMS, vol. 51(4), pages 655-661, April.
    14. Csató, László, 2019. "A characterization of the Logarithmic Least Squares Method," European Journal of Operational Research, Elsevier, vol. 276(1), pages 212-216.
    15. Vladimír Bureš & Daniela Ponce & Pavel Čech & Karel Mls, 2019. "The effect of trial repetition and problem size on the consistency of decision making," PLOS ONE, Public Library of Science, vol. 14(5), pages 1-14, May.
    16. Gass, S. I. & Rapcsak, T., 2004. "Singular value decomposition in AHP," European Journal of Operational Research, Elsevier, vol. 154(3), pages 573-584, May.
    17. Vargas, Luis G., 2008. "The consistency index in reciprocal matrices: Comparison of deterministic and statistical approaches," European Journal of Operational Research, Elsevier, vol. 191(2), pages 454-463, December.
    18. Liu, Fang & Zou, Shu-Cai & Li, Qing, 2020. "Deriving priorities from pairwise comparison matrices with a novel consistency index," Applied Mathematics and Computation, Elsevier, vol. 374(C).
    19. Michal Gluszak & Remigiusz Gawlik & Malgorzata Zieba, 2019. "Smart and Green Buildings Features in the Decision-Making Hierarchy of Office Space Tenants: An Analytic Hierarchy Process Study," Administrative Sciences, MDPI, vol. 9(3), pages 1-16, July.
    20. Siraj, Sajid & Mikhailov, Ludmil & Keane, John A., 2015. "Contribution of individual judgments toward inconsistency in pairwise comparisons," European Journal of Operational Research, Elsevier, vol. 242(2), pages 557-567.
    21. Jiří Mazurek, 2018. "Some notes on the properties of inconsistency indices in pairwise comparisons," Operations Research and Decisions, Wroclaw University of Science and Technology, Faculty of Management, vol. 28(1), pages 27-42.
    22. Leung, L. C. & Cao, D., 2000. "On consistency and ranking of alternatives in fuzzy AHP," European Journal of Operational Research, Elsevier, vol. 124(1), pages 102-113, July.
    23. Changsheng Lin & Gang Kou & Daji Ergu, 2014. "A statistical approach to measure the consistency level of the pairwise comparison matrix," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 65(9), pages 1380-1386, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Badreya Gharib Khamis Mohammed Alblooshi & Syed Zamberi Ahmad & Matloub Hussain & Sanjay Kumar Singh, 2022. "Sustainable management of electronic waste: Empirical evidences from a stakeholders' perspective," Business Strategy and the Environment, Wiley Blackwell, vol. 31(4), pages 1856-1874, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Paul Thaddeus Kazibudzki, 2016. "An examination of performance relations among selected consistency measures for simulated pairwise judgments," Annals of Operations Research, Springer, vol. 244(2), pages 525-544, September.
    2. Vladimír Bureš & Daniela Ponce & Pavel Čech & Karel Mls, 2019. "The effect of trial repetition and problem size on the consistency of decision making," PLOS ONE, Public Library of Science, vol. 14(5), pages 1-14, May.
    3. Juan Aguarón & María Teresa Escobar & José María Moreno-Jiménez & Alberto Turón, 2020. "The Triads Geometric Consistency Index in AHP-Pairwise Comparison Matrices," Mathematics, MDPI, vol. 8(6), pages 1-17, June.
    4. Lundy, Michele & Siraj, Sajid & Greco, Salvatore, 2017. "The mathematical equivalence of the “spanning tree” and row geometric mean preference vectors and its implications for preference analysis," European Journal of Operational Research, Elsevier, vol. 257(1), pages 197-208.
    5. Sangeeta Pant & Anuj Kumar & Mangey Ram & Yury Klochkov & Hitesh Kumar Sharma, 2022. "Consistency Indices in Analytic Hierarchy Process: A Review," Mathematics, MDPI, vol. 10(8), pages 1-15, April.
    6. Brunelli, Matteo & Fedrizzi, Michele, 2024. "Inconsistency indices for pairwise comparisons and the Pareto dominance principle," European Journal of Operational Research, Elsevier, vol. 312(1), pages 273-282.
    7. Ágoston, Kolos Csaba & Csató, László, 2022. "Inconsistency thresholds for incomplete pairwise comparison matrices," Omega, Elsevier, vol. 108(C).
    8. Kun Chen & Gang Kou & J. Michael Tarn & Yan Song, 2015. "Bridging the gap between missing and inconsistent values in eliciting preference from pairwise comparison matrices," Annals of Operations Research, Springer, vol. 235(1), pages 155-175, December.
    9. Jiří Mazurek & Konrad Kulakowski, 2020. "Information gap in value propositions of business models of language schools," Operations Research and Decisions, Wroclaw University of Science and Technology, Faculty of Management, vol. 30(2), pages 77-89.
    10. Zsombor Szádoczki & Sándor Bozóki & Patrik Juhász & Sergii V. Kadenko & Vitaliy Tsyganok, 2023. "Incomplete pairwise comparison matrices based on graphs with average degree approximately 3," Annals of Operations Research, Springer, vol. 326(2), pages 783-807, July.
    11. Kou, Gang & Lin, Changsheng, 2014. "A cosine maximization method for the priority vector derivation in AHP," European Journal of Operational Research, Elsevier, vol. 235(1), pages 225-232.
    12. Kułakowski, Konrad, 2018. "Inconsistency in the ordinal pairwise comparisons method with and without ties," European Journal of Operational Research, Elsevier, vol. 270(1), pages 314-327.
    13. Matteo Rossi & Gabriella Marcarelli & Antonella Ferraro & Antonio Lucadamo, 2020. "How do Calendar Anomalies Affect an Investment Choice? A Proposal of an Analytic Hierarchy Process Model," International Journal of Economics and Financial Issues, Econjournals, vol. 10(1), pages 244-249.
    14. Pietro Amenta & Alessio Ishizaka & Antonio Lucadamo & Gabriella Marcarelli & Vijay Vyas, 2020. "Computing a common preference vector in a complex multi-actor and multi-group decision system in Analytic Hierarchy Process context," Annals of Operations Research, Springer, vol. 284(1), pages 33-62, January.
    15. László Csató, 2018. "Characterization of an inconsistency ranking for pairwise comparison matrices," Annals of Operations Research, Springer, vol. 261(1), pages 155-165, February.
    16. László Csató, 2019. "Axiomatizations of inconsistency indices for triads," Annals of Operations Research, Springer, vol. 280(1), pages 99-110, September.
    17. Liu, Fang & Zou, Shu-Cai & Li, Qing, 2020. "Deriving priorities from pairwise comparison matrices with a novel consistency index," Applied Mathematics and Computation, Elsevier, vol. 374(C).
    18. Matteo Brunelli, 2017. "Studying a set of properties of inconsistency indices for pairwise comparisons," Annals of Operations Research, Springer, vol. 248(1), pages 143-161, January.
    19. Bice Cavallo, 2019. "Coherent weights for pairwise comparison matrices and a mixed-integer linear programming problem," Journal of Global Optimization, Springer, vol. 75(1), pages 143-161, September.
    20. Liang, Fuqi & Brunelli, Matteo & Rezaei, Jafar, 2020. "Consistency issues in the best worst method: Measurements and thresholds," Omega, Elsevier, vol. 96(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:8:y:2020:i:12:p:2200-:d:459980. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.