The Customer Is Always Right? How LLMs and Humans Judge Discrimination

This paper studies how large language models (LLMs) trade off moral norms against economic incentives in discriminatory hiring decisions. Bridging discrimination economics and the literature on the moral alignment in computer science, we submit 18 frontier and local LLMs to the factorial vignette experiment of a published human survey that manipulates the motive of discrimination (customer taste-based versus statistical), the cost of non-discrimination, and explicit moral injunctions. We extend this design with LLM-relevant factors: model and user personas, reasoning instructions, scenario realism, and conversational memory. In line with the literature, we find that LLMs align with humans in the direction of the effects manipulated in the survey; we also find important inter-model heterogeneity. Beyond, we contribute to the literature with, to the best of our knowledge, five novel results: 1) the market-oriented motive (customer-taste) overwhelmingly sways models in favour of discrimination, much more than what happens for human respondents; 2) models are more polarised than humans in response to moral injunctions; 3) classic prompt engineering interventions (user stated motives and model personas) have a weak impact on the models’ “moral compass”; 4) post-training alignement, not scale, shape inter-model heterogeneity, which means that de-biasing is possible but must be explicitly implemented by model providers and 5) memory effects suggest that moral permissivity in the models can be induced by conversational contextual effects.