IDEAS home Printed from https://ideas.repec.org/a/spr/annopr/v326y2023i2d10.1007_s10479-021-04431-3.html
   My bibliography  Save this article

Inconsistency indices in pairwise comparisons: an improvement of the Consistency Index

Author

Listed:
  • Yuji Sato

    (University of Nottingham)

  • Kim Hua Tan

    (University of Nottingham)

Abstract

The Consistency Index and the Consistency Ratio of the analytic hierarchy process (AHP) were designed to measure the ratio of inconsistent judgments among pairwise comparisons (PCs), which have been the principal indices for the past four decades. Definitions of inconsistency measures for PCs have yet to be established, however, because of the difficulty in quantifying subjectivity in judgments. Therefore, an empirical review that can take such subjective factors into account is essential. In this paper, the Consistency Ratio is thus reviewed using subjective data, and then a new inconsistency index for PCs is proposed based on the review. The review is based on subjective data obtained from two opinion surveys, which focuses on the relationship between the Consistency Ratio and two indicators: (1) the conformity of the results of the AHP and that of the ranking method, and (2) the goodness-of-fit of weight elicited by the AHP to human perception. A new inconsistency index is then proposed based on the mathematical property of a pairwise comparison matrix and further validated based on the conformity and the goodness-of-fit of weight. The results show that the proposed index detects inconsistency among real-world PCs more sensitively than could the Consistency Ratio; the index might suggest the reliability of the output of a pairwise comparison matrix.

Suggested Citation

  • Yuji Sato & Kim Hua Tan, 2023. "Inconsistency indices in pairwise comparisons: an improvement of the Consistency Index," Annals of Operations Research, Springer, vol. 326(2), pages 809-830, July.
    • Handle: RePEc:spr:annopr:v:326:y:2023:i:2:d:10.1007_s10479-021-04431-3
    DOI: 10.1007/s10479-021-04431-3
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10479-021-04431-3
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10479-021-04431-3?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Dong, Yucheng & Hong, Wei-Chiang & Xu, Yinfeng & Yu, Shui, 2013. "Numerical scales generated individually for analytic hierarchy process," European Journal of Operational Research, Elsevier, vol. 229(3), pages 654-662.
    2. Alessio Ishizaka & Ashraf Labib, 2014. "A hybrid and integrated approach to evaluate and prevent disasters," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 65(10), pages 1475-1489, October.
    3. Athakorn Kengpol & Sopida Tuammee, 2016. "The development of a decision support framework for a quantitative risk assessment in multimodal green logistics: an empirical study," International Journal of Production Research, Taylor & Francis Journals, vol. 54(4), pages 1020-1038, February.
    4. Matteo Brunelli & Michele Fedrizzi, 2015. "Axiomatic properties of inconsistency indices for pairwise comparisons," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 66(1), pages 1-15, January.
    5. Sato, Yuji & Tan, Kim Hua & Tse, Ying Kei, 2017. "Investment performance analysis of industrial products: Case of an effluent processing facility at a chemical company," International Journal of Production Economics, Elsevier, vol. 194(C), pages 52-58.
    6. Szybowski, Jacek & Kułakowski, Konrad & Prusak, Anna, 2020. "New inconsistency indicators for incomplete pairwise comparisons matrices," Mathematical Social Sciences, Elsevier, vol. 108(C), pages 138-145.
    7. Michele Fedrizzi & Fabio Ferrari, 2018. "A chi-square-based inconsistency index for pairwise comparison matrices," Journal of the Operational Research Society, Taylor & Francis Journals, vol. 69(7), pages 1125-1134, July.
    8. Kuenz Murphy, Catherine, 1993. "Limits on the analytic hierarchy process from its consistency index," European Journal of Operational Research, Elsevier, vol. 65(1), pages 138-139, February.
    9. Saaty, Thomas L., 2006. "Rank from comparisons and from ratings in the analytic hierarchy/network processes," European Journal of Operational Research, Elsevier, vol. 168(2), pages 557-570, January.
    10. Gass, S. I. & Rapcsak, T., 2004. "Singular value decomposition in AHP," European Journal of Operational Research, Elsevier, vol. 154(3), pages 573-584, May.
    11. Stein, William E. & Mizzi, Philip J., 2007. "The harmonic consistency index for the analytic hierarchy process," European Journal of Operational Research, Elsevier, vol. 177(1), pages 488-497, February.
    12. Kou, Gang & Lin, Changsheng, 2014. "A cosine maximization method for the priority vector derivation in AHP," European Journal of Operational Research, Elsevier, vol. 235(1), pages 225-232.
    13. Yoram Wind & Thomas L. Saaty, 1980. "Marketing Applications of the Analytic Hierarchy Process," Management Science, INFORMS, vol. 26(7), pages 641-658, July.
    14. Salo, Ahti A. & Hamalainen, Raimo P., 1995. "Preference programming through approximate ratio comparisons," European Journal of Operational Research, Elsevier, vol. 82(3), pages 458-475, May.
    15. Dey, Prasanta Kumar & Bhattacharya, Arijit & Ho, William, 2015. "Strategic supplier performance evaluation: A case-based action research of a UK manufacturing organisation," International Journal of Production Economics, Elsevier, vol. 166(C), pages 192-214.
    16. Sato, Yuji & Tan, Kim Hua & Tse, Ying Kei, 2015. "An integrated marginal analysis approach for build-to-order products," International Journal of Production Economics, Elsevier, vol. 170(PB), pages 422-428.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sangeeta Pant & Anuj Kumar & Mangey Ram & Yury Klochkov & Hitesh Kumar Sharma, 2022. "Consistency Indices in Analytic Hierarchy Process: A Review," Mathematics, MDPI, vol. 10(8), pages 1-15, April.
    2. Brunelli, Matteo & Fedrizzi, Michele, 2024. "Inconsistency indices for pairwise comparisons and the Pareto dominance principle," European Journal of Operational Research, Elsevier, vol. 312(1), pages 273-282.
    3. Ho, William & Ma, Xin, 2018. "The state-of-the-art integrations and applications of the analytic hierarchy process," European Journal of Operational Research, Elsevier, vol. 267(2), pages 399-414.
    4. Liu, Fang & Zou, Shu-Cai & Li, Qing, 2020. "Deriving priorities from pairwise comparison matrices with a novel consistency index," Applied Mathematics and Computation, Elsevier, vol. 374(C).
    5. Jiří Mazurek, 2018. "Some notes on the properties of inconsistency indices in pairwise comparisons," Operations Research and Decisions, Wroclaw University of Science and Technology, Faculty of Management, vol. 28(1), pages 27-42.
    6. Juan Aguarón & María Teresa Escobar & José María Moreno-Jiménez & Alberto Turón, 2020. "The Triads Geometric Consistency Index in AHP-Pairwise Comparison Matrices," Mathematics, MDPI, vol. 8(6), pages 1-17, June.
    7. László Csató, 2019. "Axiomatizations of inconsistency indices for triads," Annals of Operations Research, Springer, vol. 280(1), pages 99-110, September.
    8. Matteo Brunelli, 2017. "Studying a set of properties of inconsistency indices for pairwise comparisons," Annals of Operations Research, Springer, vol. 248(1), pages 143-161, January.
    9. Brunelli, Matteo & Fedrizzi, Michele, 2015. "Boundary properties of the inconsistency of pairwise comparisons in group decisions," European Journal of Operational Research, Elsevier, vol. 240(3), pages 765-773.
    10. József Temesi, 2019. "An interactive approach to determine the elements of a pairwise comparison matrix," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 27(2), pages 533-549, June.
    11. Kuei-Hu Chang & Yung-Chia Chang & Kai Chain & Hsiang-Yu Chung, 2016. "Integrating Soft Set Theory and Fuzzy Linguistic Model to Evaluate the Performance of Training Simulation Systems," PLOS ONE, Public Library of Science, vol. 11(9), pages 1-29, September.
    12. Liu Fang & Peng Yanan & Zhang Weiguo & Pedrycz Witold, 2017. "On Consistency in AHP and Fuzzy AHP," Journal of Systems Science and Information, De Gruyter, vol. 5(2), pages 128-147, April.
    13. József Temesi, 2011. "Pairwise comparison matrices and the error-free property of the decision maker," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 19(2), pages 239-249, June.
    14. Matteo Brunelli & Luisa Canal & Michele Fedrizzi, 2013. "Inconsistency indices for pairwise comparison matrices: a numerical study," Annals of Operations Research, Springer, vol. 211(1), pages 493-509, December.
    15. Sándor Bozóki & Linda Dezső & Attila Poesz & József Temesi, 2013. "Analysis of pairwise comparison matrices: an empirical research," Annals of Operations Research, Springer, vol. 211(1), pages 511-528, December.
    16. Chao, Xiangrui & Kou, Gang & Li, Tie & Peng, Yi, 2018. "Jie Ke versus AlphaGo: A ranking approach using decision making method for large-scale data with incomplete information," European Journal of Operational Research, Elsevier, vol. 265(1), pages 239-247.
    17. Kou, Gang & Lin, Changsheng, 2014. "A cosine maximization method for the priority vector derivation in AHP," European Journal of Operational Research, Elsevier, vol. 235(1), pages 225-232.
    18. Li, Kevin W. & Wang, Zhou-Jing & Tong, Xiayu, 2016. "Acceptability analysis and priority weight elicitation for interval multiplicative comparison matrices," European Journal of Operational Research, Elsevier, vol. 250(2), pages 628-638.
    19. Amenta, Pietro & Lucadamo, Antonio & Marcarelli, Gabriella, 2021. "On the choice of weights for aggregating judgments in non-negotiable AHP group decision making," European Journal of Operational Research, Elsevier, vol. 288(1), pages 294-301.
    20. Yeh, Chung-Hsing & Chang, Yu-Hern, 2009. "Modeling subjective evaluation for fuzzy group multicriteria decision making," European Journal of Operational Research, Elsevier, vol. 194(2), pages 464-473, April.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:annopr:v:326:y:2023:i:2:d:10.1007_s10479-021-04431-3. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.