Public Opinion on Fairness and Efficiency for Algorithmic and Human Decision-Makers

This study explores the public's preferences between algorithmic and human decision-makers (DMs) in high-stakes contexts, how these preferences are impacted by performance metrics, and whether the public's evaluation of performance differs when considering algorithmic versus human DMs. Leveraging a conjoint experimental design, respondents (n = 9,030) chose between pairs of DM profiles in two scenarios: pre-trial release decisions and bank loan decisions. DM profiles varied on the DM’s type (human v. algorithm) and on three metrics—defendant crime rate/loan default rate, false positive rate (FPR) among white defendants/applicants, and FPR among minority defendants/applicants—as well as an implicit fairness metric defined by the absolute difference between the two FPRs. Controlling for performance, we observe a general tendency to favor human DMs, though this is driven by a subset of respondents who expect human DMs to perform better in the real world. In addition, although a large portion of respondents claimed to prioritize fairness, we find that the impact of fairness on respondents' actual choices is limited. We also find that the relative importance of the four performance metrics remains consistent across DM type, suggesting that the public's preferences related to DM performance do not vary fundamentally between algorithmic and human DMs. Taken together, our analysis suggests that the public as a whole does not hold algorithmic DMs to a stricter fairness or efficiency standard, which has important implications as policymakers and technologists grapple with the integration of AI into pivotal societal functions.