Ranking Model Averaging: Ranking Based on Model Averaging

Ranking problems are commonly encountered in practical applications, including order priority ranking, wine quality ranking, and piston slap noise performance ranking. The responses of these ranking applications are often considered as continuous responses, and there is uncertainty on which scoring function is used to model the responses. In this paper, we address the scoring function uncertainty of continuous response ranking problems by proposing a ranking model averaging (RMA) method. With a set of candidate models varied by scoring functions, RMA assigns weights for each model determined by a K -fold crossvalidation criterion based on pairwise loss. We provide two main theoretical properties for RMA. First, we prove that the averaging ranking predictions of RMA are asymptotically optimal in achieving the lowest possible ranking risk. Second, we provide a bound on the difference between the empirical RMA weights and theoretical optimal ones, and we show that RMA weights are consistent. Simulation results validate RMA superiority over competing methods in reducing ranking risk. Moreover, when applied to empirical examples—order priority, wine quality, and piston slap noise—RMA shows its effectiveness in building accurate ranking systems.