IDEAS home Printed from https://ideas.repec.org/a/spr/annopr/v325y2023i2d10.1007_s10479-022-04682-8.html
   My bibliography  Save this article

Evaluating scales for pairwise comparisons

Author

Listed:
  • Bice Cavallo

    (University of Naples Federico II)

  • Alessio Ishizaka

    (NEOMA Business School)

Abstract

Pairwise comparisons have been a long-standing technique for comparing alternatives/criteria and their role has been pivotal in the development of modern decision-making methods. The evaluation is very often done linguistically. Several scales have been proposed to translate the linguistic evaluation into a quantitative evaluation. In this paper, we perform an experiment to investigate, under our methodological choices, which type of scale provides the best matching of the decision-maker’s verbal representation. The experiment aims to evaluate the suitability of eight evaluation scales for problems of different sizes. We find that the inverse linear scale provides the best matching verbal representation whenever the objective data are measured by means of pairwise comparisons matrices and a suitable distance between matrices is applied for computing the matching error.

Suggested Citation

  • Bice Cavallo & Alessio Ishizaka, 2023. "Evaluating scales for pairwise comparisons," Annals of Operations Research, Springer, vol. 325(2), pages 951-965, June.
    • Handle: RePEc:spr:annopr:v:325:y:2023:i:2:d:10.1007_s10479-022-04682-8
    DOI: 10.1007/s10479-022-04682-8
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10479-022-04682-8
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10479-022-04682-8?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Dong, Yucheng & Hong, Wei-Chiang & Xu, Yinfeng & Yu, Shui, 2013. "Numerical scales generated individually for analytic hierarchy process," European Journal of Operational Research, Elsevier, vol. 229(3), pages 654-662.
    2. A Ishizaka & D Balkenborg & T Kaplan, 2011. "Influence of aggregation and measurement scale on ranking a compromise alternative in AHP," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 62(4), pages 700-710, April.
    3. László Csató, 2019. "Axiomatizations of inconsistency indices for triads," Annals of Operations Research, Springer, vol. 280(1), pages 99-110, September.
    4. Sándor Bozóki & Linda Dezső & Attila Poesz & József Temesi, 2013. "Analysis of pairwise comparison matrices: an empirical research," Annals of Operations Research, Springer, vol. 211(1), pages 511-528, December.
    5. Lootsma, F. A., 1989. "Conflict resolution via pairwise comparison of concessions," European Journal of Operational Research, Elsevier, vol. 40(1), pages 109-116, May.
    6. László Csató, 2018. "Characterization of an inconsistency ranking for pairwise comparison matrices," Annals of Operations Research, Springer, vol. 261(1), pages 155-165, February.
    7. Ralph L. Keeney & Detlof von Winterfeldt & Thomas Eppel, 1990. "Eliciting Public Values for Complex Policy Decisions," Management Science, INFORMS, vol. 36(9), pages 1011-1030, September.
    8. Elliott, Michael A., 2010. "Selecting numerical scales for pairwise comparisons," Reliability Engineering and System Safety, Elsevier, vol. 95(7), pages 750-763.
    9. Bice Cavallo & Alessio Ishizaka & Maria Grazia Olivieri & Massimo Squillante, 2019. "Comparing inconsistency of pairwise comparison matrices depending on entries," Journal of the Operational Research Society, Taylor & Francis Journals, vol. 70(5), pages 842-850, May.
    10. Patrick T. Harker & Luis G. Vargas, 1987. "The Theory of Ratio Scale Estimation: Saaty's Analytic Hierarchy Process," Management Science, INFORMS, vol. 33(11), pages 1383-1403, November.
    11. Matteo Brunelli & Bice Cavallo, 2020. "Incoherence measures and relations between coherence conditions for pairwise comparisons," Decisions in Economics and Finance, Springer;Associazione per la Matematica, vol. 43(2), pages 613-635, December.
    12. Rezaei, Jafar, 2015. "Best-worst multi-criteria decision-making method," Omega, Elsevier, vol. 53(C), pages 49-57.
    13. Liang, Liang & Wang, Guohua & Hua, Zhongsheng & Zhang, Bin, 2008. "Mapping verbal responses to numerical scales in the analytic hierarchy process," Socio-Economic Planning Sciences, Elsevier, vol. 42(1), pages 46-55, March.
    14. László Csató, 2017. "On the ranking of a Swiss system chess team tournament," Annals of Operations Research, Springer, vol. 254(1), pages 17-36, July.
    15. Ishizaka, Alessio & Siraj, Sajid, 2018. "Are multi-criteria decision-making tools useful? An experimental comparative study of three methods," European Journal of Operational Research, Elsevier, vol. 264(2), pages 462-471.
    16. Elena Rokou & Konstantinos Kirytopoulos, 2014. "A Calibrated Group Decision Process," Group Decision and Negotiation, Springer, vol. 23(6), pages 1369-1384, November.
    17. Alessio Ishizaka & Sajid Siraj, 2020. "Interactive consistency correction in the analytic hierarchy process to preserve ranks," Decisions in Economics and Finance, Springer;Associazione per la Matematica, vol. 43(2), pages 443-464, December.
    18. Linares, Pedro, 2009. "Are inconsistent decisions better? An experiment with pairwise comparisons," European Journal of Operational Research, Elsevier, vol. 193(2), pages 492-498, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sangeeta Pant & Anuj Kumar & Mangey Ram & Yury Klochkov & Hitesh Kumar Sharma, 2022. "Consistency Indices in Analytic Hierarchy Process: A Review," Mathematics, MDPI, vol. 10(8), pages 1-15, April.
    2. Klaus D. Goepel, 2019. "Comparison of Judgment Scales of the Analytical Hierarchy Process — A New Approach," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 18(02), pages 445-463, March.
    3. Zsombor Szádoczki & Sándor Bozóki & Patrik Juhász & Sergii V. Kadenko & Vitaliy Tsyganok, 2023. "Incomplete pairwise comparison matrices based on graphs with average degree approximately 3," Annals of Operations Research, Springer, vol. 326(2), pages 783-807, July.
    4. Alessio Ishizaka & Sajid Siraj, 2020. "Interactive consistency correction in the analytic hierarchy process to preserve ranks," Decisions in Economics and Finance, Springer;Associazione per la Matematica, vol. 43(2), pages 443-464, December.
    5. A Ishizaka & D Balkenborg & T Kaplan, 2011. "Does AHP help us make a choice? An experimental evaluation," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 62(10), pages 1801-1812, October.
    6. Abel, Edward & Mikhailov, Ludmil & Keane, John, 2018. "Inconsistency reduction in decision making via multi-objective optimisation," European Journal of Operational Research, Elsevier, vol. 267(1), pages 212-226.
    7. Sérgio J Teixeira & João J Ferreira & Peter Wanke & Jorge Junio Moreira Antunes, 2021. "Evaluation model of competitive and innovative tourism practices based on information entropy and alternative criteria weight," Tourism Economics, , vol. 27(1), pages 23-44, February.
    8. Corrente, S. & Figueira, J.R. & Greco, S., 2021. "Pairwise comparison tables within the deck of cards method in multiple criteria decision aiding," European Journal of Operational Research, Elsevier, vol. 291(2), pages 738-756.
    9. Wanke, Peter Fernandes & Chiappetta Jabbour, Charbel José & Moreira Antunes, Jorge Junio & Lopes de Sousa Jabbour, Ana Beatriz & Roubaud, David & Sobreiro, Vinicius Amorim & Santibanez Gonzalez‬, Erne, 2021. "An original information entropy-based quantitative evaluation model for low-carbon operations in an emerging market," International Journal of Production Economics, Elsevier, vol. 234(C).
    10. Siraj, Sajid & Mikhailov, Ludmil & Keane, John A., 2015. "Contribution of individual judgments toward inconsistency in pairwise comparisons," European Journal of Operational Research, Elsevier, vol. 242(2), pages 557-567.
    11. Paweł Karczmarek & Witold Pedrycz & Adam Kiersztyn, 2021. "Fuzzy Analytic Hierarchy Process in a Graphical Approach," Group Decision and Negotiation, Springer, vol. 30(2), pages 463-481, April.
    12. Siraj, S. & Mikhailov, L. & Keane, J.A., 2012. "Preference elicitation from inconsistent judgments using multi-objective optimization," European Journal of Operational Research, Elsevier, vol. 220(2), pages 461-471.
    13. Corrente, Salvatore & Greco, Salvatore & Ishizaka, Alessio, 2016. "Combining analytical hierarchy process and Choquet integral within non-additive robust ordinal regression," Omega, Elsevier, vol. 61(C), pages 2-18.
    14. Szádoczki, Zsombor & Bozóki, Sándor & Tekile, Hailemariam Abebe, 2022. "Filling in pattern designs for incomplete pairwise comparison matrices: (Quasi-)regular graphs with minimal diameter," Omega, Elsevier, vol. 107(C).
    15. Michele Fedrizzi & Nino Civolani & Andrew Critch, 2020. "Inconsistency evaluation in pairwise comparison using norm-based distances," Decisions in Economics and Finance, Springer;Associazione per la Matematica, vol. 43(2), pages 657-672, December.
    16. Brunelli, Matteo & Fedrizzi, Michele, 2024. "Inconsistency indices for pairwise comparisons and the Pareto dominance principle," European Journal of Operational Research, Elsevier, vol. 312(1), pages 273-282.
    17. Vladimír Bureš & Daniela Ponce & Pavel Čech & Karel Mls, 2019. "The effect of trial repetition and problem size on the consistency of decision making," PLOS ONE, Public Library of Science, vol. 14(5), pages 1-14, May.
    18. Ergu, Daji & Kou, Gang & Peng, Yi & Shi, Yong, 2011. "A simple method to improve the consistency ratio of the pair-wise comparison matrix in ANP," European Journal of Operational Research, Elsevier, vol. 213(1), pages 246-259, August.
    19. A Ishizaka & D Balkenborg & T Kaplan, 2011. "Influence of aggregation and measurement scale on ranking a compromise alternative in AHP," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 62(4), pages 700-710, April.
    20. Mangirdas Morkunas & Elzė Rudienė & Lukas Giriūnas & Laura Daučiūnienė, 2020. "Assessment of Factors Causing Bias in Marketing- Related Publications," Publications, MDPI, vol. 8(4), pages 1-16, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:annopr:v:325:y:2023:i:2:d:10.1007_s10479-022-04682-8. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.