IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v285y2020i3p1025-1041.html
   My bibliography  Save this article

A new correlation coefficient for comparing and aggregating non-strict and incomplete rankings

Author

Listed:
  • Yoo, Yeawon
  • Escobedo, Adolfo R.
  • Skolfield, J. Kyle

Abstract

We introduce a correlation coefficient that is designed to deal with a variety of ranking formats including those containing non-strict (i.e., with-ties) and incomplete (i.e., unknown) preferences. The correlation coefficient is designed to enforce a neutral treatment of incompleteness whereby no assumptions are made about individual preferences involving unranked objects. The new measure, which can be regarded as a generalization of the seminal Kendall tau correlation coefficient, is proven to satisfy a set of metric-like axioms and to be equivalent to a recently developed ranking distance function associated with Kemeny aggregation. In an effort to further unify and enhance both robust ranking methodologies, this work proves the equivalence of an additional distance and correlation-coefficient pairing in the space of non-strict incomplete rankings. These connections induce new exact optimization methodologies: a specialized branch and bound algorithm and an exact integer programming formulation. Moreover, the bridging of these complementary theories reinforces the singular suitability of the featured correlation coefficient to solve the general consensus ranking problem. The latter premise is bolstered by an accompanying set of experiments on random instances, which are generated via a herein developed sampling technique connected with the classic Mallows distribution of ranking data. Associated experiments with the branch and bound algorithm demonstrate that, as data becomes noisier, the featured correlation coefficient yields relatively fewer alternative optimal solutions and that the aggregate rankings tend to be closer to an underlying ground truth shared by a majority.

Suggested Citation

  • Yoo, Yeawon & Escobedo, Adolfo R. & Skolfield, J. Kyle, 2020. "A new correlation coefficient for comparing and aggregating non-strict and incomplete rankings," European Journal of Operational Research, Elsevier, vol. 285(3), pages 1025-1041.
    • Handle: RePEc:eee:ejores:v:285:y:2020:i:3:p:1025-1041
    DOI: 10.1016/j.ejor.2020.02.027
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221720301454
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ejor.2020.02.027?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Kapetanios, George & Price, Simon & Tasiou, Menelaos & Ventouri, Alexia, 2021. "State-level wage Phillips curves," Econometrics and Statistics, Elsevier, vol. 18(C), pages 1-11.
    2. Defever, Fabrice & Reyes, José-Daniel & Riaño, Alejandro & Varela, Gonzalo, 2020. "All these worlds are yours, except india: The effectiveness of cash subsidies to export in nepal," European Economic Review, Elsevier, vol. 128(C).
    3. Emara, Noha & El Said, Ayah & Pearlman, Joseph, 2019. "On the Impact of Financial Inclusion on Financial Stability and Inequality: The Role of Macroprudential Policies," MPRA Paper 99258, University Library of Munich, Germany.
    4. Lin Shili, 2010. "Space Oriented Rank-Based Data Integration," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 9(1), pages 1-25, April.
    5. Kim, Soung Hie & Ahn, Byeong Seok, 1999. "Interactive group decision making procedure under incomplete information," European Journal of Operational Research, Elsevier, vol. 116(3), pages 498-507, August.
    6. Aligishiev, Z. & Ben-Gad, M. & Pearlman, J., 2020. "Evaluating Historical Episodes using Shock Decompositions in the DSGE Model," Working Papers 20/10, Department of Economics, City University London.
    7. Bai, Y. & Girma, S. & Riaño, A., 2020. "Corporate Acquisitions and Firm-level Uncertainty: Domestic versus Cross-Border Deals," Working Papers 20/09, Department of Economics, City University London.
    8. Wade Cook & Moshe Kress & Lawrence Seiford, 1986. "Information and preference in partial orders: A bimatrix representation," Psychometrika, Springer;The Psychometric Society, vol. 51(2), pages 197-207, June.
    9. Erick Moreno-Centeno & Adolfo R. Escobedo, 2016. "Axiomatic aggregation of incomplete rankings," IISE Transactions, Taylor & Francis Journals, vol. 48(6), pages 475-488, June.
    10. Dorit S. Hochbaum & Asaf Levin, 2006. "Methodologies and Algorithms for Group-Rankings Decision," Management Science, INFORMS, vol. 52(9), pages 1394-1408, September.
    11. Jean-Paul Doignon & Aleksandar Pekeč & Michel Regenwetter, 2004. "The repeated insertion model for rankings: Missing link between two subset choice models," Psychometrika, Springer;The Psychometric Society, vol. 69(1), pages 33-54, March.
    12. Antonio D’Ambrosio & Willem J. Heiser, 2016. "A Recursive Partitioning Method for the Prediction of Preference Rankings Based Upon Kemeny Distances," Psychometrika, Springer;The Psychometric Society, vol. 81(3), pages 774-794, September.
    13. Cook, Wade D., 2006. "Distance-based and ad hoc consensus models in ordinal preference ranking," European Journal of Operational Research, Elsevier, vol. 172(2), pages 369-385, July.
    14. Bharadwaj, P. & Doiron, D. & Fiebig, D. G. & Suziedelyte, A., 2020. "Psychological Costs of Migration: Home Country Natural Disasters and Mental Health," Working Papers 20/03, Department of Economics, City University London.
    15. Young, H. P., 1988. "Condorcet's Theory of Voting," American Political Science Review, Cambridge University Press, vol. 82(4), pages 1231-1244, December.
    16. Khaled Jabeur & Jean-Marc Martel & Slim Ben Khélifa, 2004. "A Distance-Based Collective Preorder Integrating the Relative Importance of the Group's Members," Group Decision and Negotiation, Springer, vol. 13(4), pages 327-349, July.
    17. Offiaeli, K. & Yaman, F., 2020. "Social Norms as a Cost-Effective Measure of Managing Transport Demand: Evidence from an Experiment on the London Underground," Working Papers 20/07, Department of Economics, City University London.
    18. Che, X. & Huang, Y. & Zhang, L., 2020. "Supervisory Efficiency and Collusion in a Multiple-Agent Hierarchy," Working Papers 20/01, Department of Economics, City University London.
    19. R. L. Plackett, 1975. "The Analysis of Permutations," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 24(2), pages 193-202, June.
    20. Grzegorzewski, Przemyslaw, 2006. "The coefficient of concordance for vague data," Computational Statistics & Data Analysis, Elsevier, vol. 51(1), pages 314-322, November.
    21. Amodio, S. & D’Ambrosio, A. & Siciliano, R., 2016. "Accurate algorithms for identifying the median ranking when dealing with weak and partial rankings under the Kemeny axiomatic approach," European Journal of Operational Research, Elsevier, vol. 249(2), pages 667-676.
    22. Che, X. & Katayama, H. & Lee, P., 2020. "Willingness to Pay for Brand Reputation: Lessons from the Volkswagen Diesel Emissions Scandal," Working Papers 20/02, Department of Economics, City University London.
    23. Slim Ben Khelifa & Jean-Marc Martel, 2001. "A Distance-Based Collective Weak Ordering," Group Decision and Negotiation, Springer, vol. 10(4), pages 317-329, July.
    24. Erick Moreno-Centeno & Richard M. Karp, 2013. "The Implicit Hitting Set Approach to Solve Combinatorial Optimization Problems with an Application to Multigenome Alignment," Operations Research, INFORMS, vol. 61(2), pages 453-468, April.
    25. Laliotis, Ioannis & Minos, Dimitrios, 2020. "Spreading the disease: The role of culture," SocArXiv z4ndc, Center for Open Science.
    26. Wade D. Cook & Moshe Kress, 1985. "Ordinal Ranking with Intensity of Preference," Management Science, INFORMS, vol. 31(1), pages 26-32, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Akbari, Sina & Escobedo, Adolfo R., 2023. "Beyond kemeny rank aggregation: A parameterizable-penalty framework for robust ranking aggregation with ties," Omega, Elsevier, vol. 119(C).
    2. Francisco Pedroche & J. Alberto Conejero, 2020. "Corrected Evolutive Kendall’s τ Coefficients for Incomplete Rankings with Ties: Application to Case of Spotify Lists," Mathematics, MDPI, vol. 8(10), pages 1-30, October.
    3. Yeawon Yoo & Adolfo R. Escobedo, 2021. "A New Binary Programming Formulation and Social Choice Property for Kemeny Rank Aggregation," Decision Analysis, INFORMS, vol. 18(4), pages 296-320, December.
    4. Adolfo R. Escobedo & Romena Yasmin, 2023. "Derivations of large classes of facet defining inequalities of the weak order polytope using ranking structures," Journal of Combinatorial Optimization, Springer, vol. 46(3), pages 1-45, October.
    5. Baak, M. & Koopman, R. & Snoek, H. & Klous, S., 2020. "A new correlation coefficient between categorical, ordinal and interval variables with Pearson characteristics," Computational Statistics & Data Analysis, Elsevier, vol. 152(C).
    6. Fu, Yelin & Lu, Yihe & Yu, Chen & Lai, Kin Keung, 2022. "Inter-country comparisons of energy system performance with the energy trilemma index: An ensemble ranking methodology based on the half-quadratic theory," Energy, Elsevier, vol. 261(PA).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Nie, Binjian & Zou, Boyang & She, Xiaohui & Zhang, Tongtong & Li, Yongliang & Ding, Yulong, 2020. "Development of a heat transfer coefficient based design method of a thermal energy storage device for transport air-conditioning applications," Energy, Elsevier, vol. 196(C).
    2. Li, Zhiyu & Xu, Xiwei & Jiang, Enchen & Han, Ping & Sun, Yan & Zhou, Ling & Zhong, Peidong & Fan, Xudong, 2020. "Alkane from hydrodeoxygenation (HDO) combined with in-situ multistage condensation of biomass continuous pyrolysis bio-oil via mixed supports catalyst Ni/HZSM-5-γ-Al2O3," Renewable Energy, Elsevier, vol. 149(C), pages 535-548.
    3. Zhu, H.B. & Zhou, Y.J. & Wu, W.J., 2020. "Modeling traffic flow mixed with automated vehicles considering drivers ’ character difference," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 549(C).
    4. Rotundo, G. & Ausloos, M., 2007. "Microeconomic co-evolution model for financial technical analysis signals," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 373(C), pages 569-585.
    5. Alderremy, A.A. & Saad, Khaled M. & Agarwal, Praveen & Aly, Shaban & Jain, Shilpi, 2020. "Certain new models of the multi space-fractional Gardner equation," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 545(C).
    6. Coma, Julià & Chàfer, Marta & Pérez, Gabriel & Cabeza, Luisa F., 2020. "How internal heat loads of buildings affect the effectiveness of vertical greenery systems? An experimental study," Renewable Energy, Elsevier, vol. 151(C), pages 919-930.
    7. Shi, Xin & Jiang, Haizhou & Li, Huan & Xu, Dong, 2020. "Maritime cluster research: Evolutionary classification and future development," Transportation Research Part A: Policy and Practice, Elsevier, vol. 133(C), pages 237-254.
    8. Yeawon Yoo & Adolfo R. Escobedo, 2021. "A New Binary Programming Formulation and Social Choice Property for Kemeny Rank Aggregation," Decision Analysis, INFORMS, vol. 18(4), pages 296-320, December.
    9. Yucheng Dong & Yao Li & Ying He & Xia Chen, 2021. "Preference–Approval Structures in Group Decision Making: Axiomatic Distance and Aggregation," Decision Analysis, INFORMS, vol. 18(4), pages 273-295, December.
    10. Ben Amor, Sarah & Martel, Jean-Marc, 2014. "A new distance measure including the weak preference relation: Application to the multiple criteria aggregation procedure for mixed evaluations," European Journal of Operational Research, Elsevier, vol. 237(3), pages 1165-1169.
    11. Antonio D’Ambrosio & Carmela Iorio & Michele Staiano & Roberta Siciliano, 2019. "Median constrained bucket order rank aggregation," Computational Statistics, Springer, vol. 34(2), pages 787-802, June.
    12. Jabeur, Khaled & Martel, Jean-Marc, 2007. "An ordinal sorting method for group decision-making," European Journal of Operational Research, Elsevier, vol. 180(3), pages 1272-1289, August.
    13. Akbari, Sina & Escobedo, Adolfo R., 2023. "Beyond kemeny rank aggregation: A parameterizable-penalty framework for robust ranking aggregation with ties," Omega, Elsevier, vol. 119(C).
    14. Zhang, Hengjie & Dong, Yucheng & Chiclana, Francisco & Yu, Shui, 2019. "Consensus efficiency in group decision making: A comprehensive comparative study and its optimal design," European Journal of Operational Research, Elsevier, vol. 275(2), pages 580-598.
    15. Azzini, Ivano & Munda, Giuseppe, 2020. "A new approach for identifying the Kemeny median ranking," European Journal of Operational Research, Elsevier, vol. 281(2), pages 388-401.
    16. Chen, Yen-Liang & Cheng, Li-Chen, 2009. "Mining maximum consensus sequences from group ranking data," European Journal of Operational Research, Elsevier, vol. 198(1), pages 241-251, October.
    17. Antonella Plaia & Simona Buscemi & Mariangela Sciandra, 2021. "Consensus among preference rankings: a new weighted correlation coefficient for linear and weak orderings," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 15(4), pages 1015-1037, December.
    18. Khaled Jabeur & Jean-Marc Martel & Slim Ben Khélifa, 2004. "A Distance-Based Collective Preorder Integrating the Relative Importance of the Group's Members," Group Decision and Negotiation, Springer, vol. 13(4), pages 327-349, July.
    19. Khaled Jabeur & Jean-Marc Martel, 2010. "An Agreement Index with Respect to a Consensus Preorder," Group Decision and Negotiation, Springer, vol. 19(6), pages 571-590, November.
    20. Jabeur, Khaled & Martel, Jean-Marc, 2007. "A collective choice method based on individual preferences relational systems (p.r.s.)," European Journal of Operational Research, Elsevier, vol. 177(3), pages 1549-1565, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:285:y:2020:i:3:p:1025-1041. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.