Selecting the best when selection is hard

In dynamic promotion contests, where performance measurement is noisy and constrained to be ordinal, selection of the most able agent can be improved by biasing later stages in favor of early performers. We show that even in the worst-case scenario, where external random factors swamp the difference in agents’ abilities in determining their relative performance, optimal bias is (i) strictly positive and (ii) locally insensitive to changes in the ratio of heterogeneity to noise. To explain these, arguably surprising, limiting results, we demonstrate a close relationship in the limit between optimal bias under ordinal information and the expected optimal bias when bias can be conditioned on cardinal information about relative performance. As a consequence of these two limiting properties, the simple rule of setting bias as if in the worst-case scenario achieves most of the potential gains in selective efficiency from biasing dynamic rank-order contests.