IDEAS home Printed from https://ideas.repec.org/a/spr/opsear/v61y2024i1d10.1007_s12597-023-00689-2.html
   My bibliography  Save this article

Specialists’ knowledge and cognitive stress in making pairwise comparisons

Author

Listed:
  • Matheus Pereira Libório

    (Pontifical Catholic University of Minas Gerais
    Enacom Handcrafted Technologies)

  • Petr Iakovlevitch Ekel

    (Pontifical Catholic University of Minas Gerais)

  • Patrícia Bernardes

    (Pontifical Catholic University of Minas Gerais)

  • Luiz Flávio Autran Monteiro Gomes

    (Ibmec University Center)

  • Douglas Alexandre Gomes Vieira

    (Enacom Handcrafted Technologies
    Federal Center for Technological Education of Minas Gerais)

Abstract

There is no lack of studies dealing with the consistency of evaluations performed by pairwise comparison in the decision-making literature. Mostly, these studies offer algorithms for reducing the inconsistency of evaluations and indices to measure the evaluation’s consistency degree. The focus on these two research fronts does not cover all the gaps associated with the inconsistent evaluation problem. The existing algorithms are difficult to implement and do not preserve the original evaluations since the original evaluation matrix is replaced with a new matrix. Furthermore, the inconsistency of pairwise comparison has been associated with the specialist’s bounded rationality only at the theoretical-conceptual level. This research investigates the relationship between the lack of specialist knowledge and the inconsistency of evaluations, as well as introduces an approach that ensures the evaluation’s consistency by reducing the specialist’s cognitive stress when comparing a high number of alternatives. The results reveal that the specialist’s limited knowledge about the topic does not impact the degree of consistency of the evaluations as expected. The evaluation’s consistency degree is 59% lower when the specialist does have no knowledge about the decision topic but has theoretical knowledge and experience in evaluating alternatives by pairwise comparison. This is a remarkable contribution with a high degree of universality and applicability because instructing decision-makers on the inconsistency problem is a cheaper, easier way to increase the evaluation’s consistency degree without altering the original information. Furthermore, the introduced approach reduces the number of evaluations and evaluation time by 8.0 and 7.8 times, respectively.

Suggested Citation

  • Matheus Pereira Libório & Petr Iakovlevitch Ekel & Patrícia Bernardes & Luiz Flávio Autran Monteiro Gomes & Douglas Alexandre Gomes Vieira, 2024. "Specialists’ knowledge and cognitive stress in making pairwise comparisons," OPSEARCH, Springer;Operational Research Society of India, vol. 61(1), pages 51-70, March.
  • Handle: RePEc:spr:opsear:v:61:y:2024:i:1:d:10.1007_s12597-023-00689-2
    DOI: 10.1007/s12597-023-00689-2
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s12597-023-00689-2
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s12597-023-00689-2?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Marta Kuc-Czarnecka & Samuele Lo Piano & Andrea Saltelli, 2020. "Quantitative Storytelling in the Making of a Composite Indicator," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 149(3), pages 775-802, June.
    2. Sándor Bozóki & János Fülöp & Attila Poesz, 2015. "On reducing inconsistency of pairwise comparison matrices below an acceptance threshold," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 23(4), pages 849-866, December.
    3. Issaka Dialga & Le Thi Hang Giang, 2017. "Highlighting Methodological Limitations in the Steps of Composite Indicators Construction," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 131(2), pages 441-465, March.
    4. Herrera-Viedma, E. & Herrera, F. & Chiclana, F. & Luque, M., 2004. "Some issues on consistency of fuzzy preference relations," European Journal of Operational Research, Elsevier, vol. 154(1), pages 98-109, April.
    5. Kolsoom Zamani & Hashem Omrani, 2022. "A complete information PCA-imprecise DEA approach for constructing composite indicator with interval data: an application for finding development degree of cities," International Journal of Operational Research, Inderscience Enterprises Ltd, vol. 44(4), pages 522-549.
    6. L Cherchye & W Moesen & N Rogge & T Van Puyenbroeck & M Saisana & A Saltelli & R Liska & S Tarantola, 2008. "Creating composite indicators with DEA and robustness analysis: the case of the Technology Achievement Index," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 59(2), pages 239-251, February.
    7. Tang, Ming & Liao, Huchang & Mi, Xiaomei & Lev, Benjamin & Pedrycz, Witold, 2021. "A hierarchical consensus reaching process for group decision making with noncooperative behaviors," European Journal of Operational Research, Elsevier, vol. 293(2), pages 632-642.
    8. Adrian Corcoran & Robert Gillanders, 2015. "Foreign direct investment and the ease of doing business," Review of World Economics (Weltwirtschaftliches Archiv), Springer;Institut für Weltwirtschaft (Kiel Institute for the World Economy), vol. 151(1), pages 103-126, February.
    9. Bowen Zhang & Yucheng Dong & Enrique Herrera-Viedma, 2019. "Group Decision Making with Heterogeneous Preference Structures: An Automatic Mechanism to Support Consensus Reaching," Group Decision and Negotiation, Springer, vol. 28(3), pages 585-617, June.
    10. Pinheiro-Alves, Ricardo & Zambujal-Oliveira, João, 2012. "The Ease of Doing Business Index as a tool for investment location decisions," Economics Letters, Elsevier, vol. 117(1), pages 66-70.
    11. Ergu, Daji & Kou, Gang & Peng, Yi & Shi, Yong, 2011. "A simple method to improve the consistency ratio of the pair-wise comparison matrix in ANP," European Journal of Operational Research, Elsevier, vol. 213(1), pages 246-259, August.
    12. Simeon Djankov, 2016. "The Doing Business Project: How It Started: Correspondence," Journal of Economic Perspectives, American Economic Association, vol. 30(1), pages 247-248, Winter.
    13. Rogge, Nicky, 2018. "Composite indicators as generalized benefit-of-the-doubt weighted averages," European Journal of Operational Research, Elsevier, vol. 267(1), pages 381-392.
    14. L. R. Figueiredo & E. A. Frej & G. L. Soares & P. Ya. Ekel, 2021. "Group Decision-Based Construction of Scenarios for Multicriteria Analysis in Conditions of Uncertainty on the Basis of Quantitative and Qualitative Information," Group Decision and Negotiation, Springer, vol. 30(3), pages 665-696, June.
    15. Nathan C. Proudlove & Mhorag Goff & Kieran Walshe & Ruth Boaden, 2019. "The signal in the noise: Robust detection of performance “outliers” in health services," Journal of the Operational Research Society, Taylor & Francis Journals, vol. 70(7), pages 1102-1114, July.
    16. Salvatore Greco & Alessio Ishizaka & Menelaos Tasiou & Gianpiero Torrisi, 2019. "On the Methodological Framework of Composite Indices: A Review of the Issues of Weighting, Aggregation, and Robustness," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 141(1), pages 61-94, January.
    17. Samira El Gibari & Trinidad Gómez & Francisco Ruiz, 2019. "Building composite indicators using multicriteria methods: a review," Journal of Business Economics, Springer, vol. 89(1), pages 1-24, February.
    18. Fusco, Elisa, 2023. "Potential improvements approach in composite indicators construction: The Multi-directional Benefit of the Doubt model," Socio-Economic Planning Sciences, Elsevier, vol. 85(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Matheus Pereira Libório & Alexandre Magno Alvez Diniz & Angélica Cidália Gouveia Santos & Cristiane Neri Nobre & Douglas Alexandre Gomes Vieira & Hasheem Mannan & Marcos Flávio Silveira Vasconcelos Da, 2024. "Benefit-of-the-Doubt in the Spatial Analysis of Child Well-Being in European Countries," Child Indicators Research, Springer;The International Society of Child Indicators (ISCI), vol. 17(4), pages 1851-1870, August.
    2. Cantone, Giulio Giacomo & Tomaselli, Venera, 2024. "On the Coherence of Composite Indexes: Multiversal Model and Specification Analysis for an Index of Well-Being," MetaArXiv d5y26, Center for Open Science.
    3. Matheus Pereira Libório & Lívia Maria Leite Silva & Petr Iakovlevitch Ekel & Letícia Ribeiro Figueiredo & Patrícia Bernardes, 2022. "Consensus-Based Sub-Indicator Weighting Approach: Constructing Composite Indicators Compatible with Expert Opinion," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 164(3), pages 1073-1099, December.
    4. Matheus Pereira Libório & Elisa Fusco & Alexandre Magno Alves Diniz & Oséias da Silva Martinuci & Petr Iakovlevitch Ekel, 2024. "A Novel Approach for Multispatial and Multitemporal Analysis of Composite Indicators," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 173(3), pages 783-800, July.
    5. repec:osf:metaar:d5y26_v1 is not listed on IDEAS
    6. Milica Maricic & Jose A. Egea & Veljko Jeremic, 2019. "A Hybrid Enhanced Scatter Search—Composite I-Distance Indicator (eSS-CIDI) Optimization Approach for Determining Weights Within Composite Indicators," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 144(2), pages 497-537, July.
    7. Panagiotis Artelaris, 2022. "A development index for the Greek regions," Quality & Quantity: International Journal of Methodology, Springer, vol. 56(3), pages 1261-1281, June.
    8. Matheus Pereira Libório & Petr Yakovlevitch Ekel & Oseias da Silva Martinuci & Letícia Ribeiro Figueiredo & Renato Moreira Hadad & Renata de Mello Lyrio & Patrícia Bernardes, 2022. "Fuzzy set based intra-urban inequality indicator," Quality & Quantity: International Journal of Methodology, Springer, vol. 56(2), pages 667-687, April.
    9. Alexei Manso Correa Machado & Petr Iakovlevitch Ekel & Matheus Pereira Libório, 2023. "Goal-based participatory weighting scheme: balancing objectivity and subjectivity in the construction of composite indicators," Quality & Quantity: International Journal of Methodology, Springer, vol. 57(5), pages 4387-4407, October.
    10. Rogge, Nicky & Archer, Geoffrey, 2021. "Measuring and analyzing country change in establishing ease of doing business using a revised version of World Bank's ease of doing business index," European Journal of Operational Research, Elsevier, vol. 290(1), pages 373-385.
    11. Matheus Pereira Libório & João Francisco Abreu & Petr Iakovlevitch Ekel & Alexei Manso Correa Machado, 2023. "Effect of sub-indicator weighting schemes on the spatial dependence of multidimensional phenomena," Journal of Geographical Systems, Springer, vol. 25(2), pages 185-211, April.
    12. Matheus Pereira Libório & Alexandre Magno Alvez Diniz & Douglas Alexandre Gomes Vieira & Petr Iakovlevitch Ekel, 2024. "Subjective–Objective Method of Maximizing the Average Variance Extracted From Sub-indicators in Composite Indicators," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 175(2), pages 613-637, November.
    13. Matheus Pereira Libório & Oseias da Silva Martinuci & Alexei Manso Correa Machado & Renata de Mello Lyrio & Patrícia Bernardes, 2022. "Time–Space Analysis of Multidimensional Phenomena: A Composite Indicator of Social Exclusion Through k-Means," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 159(2), pages 569-591, January.
    14. Patrícia Bernardes & Petr Iakovlevitch Ekel & Sérgio Fernando Loureiro Rezende & Joel Gomes Pereira Júnior & Angélica Cidália Gouveia Santos & Maurício Andrade Rodrigues Costa & Rafael Lopes Carvalhai, 2022. "Cost of doing business index in Latin America," Quality & Quantity: International Journal of Methodology, Springer, vol. 56(4), pages 2233-2252, August.
    15. Milica Maricic & Veljko Jeremic, 2023. "Imposing unsupervised constraints to the Benefit-of-the-Doubt (BoD) model," METRON, Springer;Sapienza Università di Roma, vol. 81(3), pages 259-296, December.
    16. Liang, Fuqi & Brunelli, Matteo & Rezaei, Jafar, 2020. "Consistency issues in the best worst method: Measurements and thresholds," Omega, Elsevier, vol. 96(C).
    17. Khatab Alqararah, 2023. "Assessing the robustness of composite indicators: the case of the Global Innovation Index," Journal of Innovation and Entrepreneurship, Springer, vol. 12(1), pages 1-22, December.
    18. Brad Bottoms & Julie Arbit & Earl Lewis & Alford Young Jr, 2024. "Towards urban place-based resilience modeling: Mixed methods for a flood resilience assessment index," Environment and Planning B, , vol. 51(7), pages 1563-1580, September.
    19. Giuliano Resce & Fritz Schiltz, 2021. "Sustainable Development in Europe: A Multicriteria Decision Analysis," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 67(2), pages 509-529, June.
    20. Christoph Morosoli & Peter Draper & Andreas Freytag & Sebastian Schuhmann, 2024. "Drivers of Inclusive Development: An Empirical Investigation," The European Journal of Development Research, Palgrave Macmillan;European Association of Development Research and Training Institutes (EADI), vol. 36(4), pages 987-1015, August.
    21. Rubén Raedo, 2021. "Urban Sustainability Deficits: The Urban Non-Sustainability Index (UNSI) as a Tool for Urban Policy," Sustainability, MDPI, vol. 13(22), pages 1-25, November.

    More about this item

    Keywords

    Pairwise comparison; Matrix consistency; Preference formats; Transformation functions; Composite indicator; Group decision-making;
    All these keywords.

    JEL classification:

    • C02 - Mathematical and Quantitative Methods - - General - - - Mathematical Economics
    • C3 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables
    • C43 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - Index Numbers and Aggregation
    • C44 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - Operations Research; Statistical Decision Theory

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:opsear:v:61:y:2024:i:1:d:10.1007_s12597-023-00689-2. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.