AI Model Design and Physician Liability

Clinical algorithms can perform differently across patient groups, a concern that grows as these algorithms increasingly take the form of artificial intelligence (AI) models. In the United States, a 2024 federal rule bars healthcare providers that receive federal funds from discriminating through their use of clinical algorithms, which exposes them to liability when such use disadvantages a protected patient group. We examine how such liability considerations reshape (i) an AI firm's model design decisions that drive group-specific accuracy and (ii) a physician's decisions to use AI in healthcare delivery. The firm designs an AI model for two patient groups, and improving accuracy for the disadvantaged group is more costly. The physician (who remains the accountable decision maker) then decides whether to consult AI, weighing the reduction in clinical uncertainty against expected liability exposure when AI errors disproportionately affect the disadvantaged group. We find that this liability can induce disparate use of AI: the physician may reduce AI use overall and, over an intermediate range of liability, rely on AI less for disadvantaged patients. The effect is non-monotone. As liability increases, the physician's use of AI for disadvantaged patients first declines, then rises as the firm reallocates investment toward reducing disparity or switches to an equal-accuracy design. Mandating equal algorithmic accuracy across patient groups can then inadvertently harm both groups, because a uniform accuracy requirement distorts the firm's investment incentives and the physician's equilibrium AI-use decisions.