Examination of the similarity between a new Sigmoid function-based consensus ranking method and four commonly-used algorithms

The problem of aggregating individual rankings to create an overall consensus ranking representative of the group is of longstanding interest in group decision making. The problem arises in situations where a group of k Decision Makers (DMs) are asked to rank order n alternatives. The question is how to combine the DMs' rankings into one consensus ranking. Several different approaches have been suggested to aggregate DM responses into a compromise or consensus ranking, however, none is generally recognised as being the best and the similarity of consensus rankings generated by these algorithms is largely unknown. In this paper, we propose a new Weighted-sum ordinal Consensus ranking Method (WCM) with the weights derived from a Sigmoid function. We run Monte Carlo simulation across a range of k and n to compare the similarity of the consensus rankings generated by our method with the best-known method of Borda–Kendall (BAK; Kendall, M. (1962) Rank correlation methods. New York, NY: Hafner) and two other commonly used techniques proposed by Beck, M.P. and Lin, B.W. (1983) 'Some heuristics for the consensus ranking problem', Computers and Operations Research, Vol. 10, pp.1–7 and Cook, W.D. and Kress, M. (1985) 'Ordinal rankings with intensity of preference', Management Science, Vol. 31, pp.26–32. WCM and BAK yielded the most similar consensus rankings (mean tau-x = .91). As the number of alternatives to be ranked increased, the similarity of rankings generated by the four algorithms decreased. Although consensus rankings generated by different algorithms were similar, differences in rankings among the algorithms were of sufficient magnitude that they often cannot be viewed as interchangeable from a practical perspective.