Author
Listed:
- David Autor
- Andrew Caplin
- Daniel J. Martin
- Philip Marx
Abstract
The cost of error in many high-stakes settings is asymmetric: misdiagnosing pneumonia when absent is an inconvenience but failing to detect it when present can be life-threatening. Accordingly, artificial intelligence (AI) models used to assist such decisions are frequently trained with asymmetric loss functions that incorporate human decision-makers' trade-offs between false positives and false negatives. In two focal applications, we show that this standard alignment practice can backfire. In both cases, it would be better to train the machine learning model with a loss function that ignores the humana??s objective and then adjust predictions ex post according to that objective. We rationalize this result using an economic model of incentive design with endogenous information acquisition. The key insight from our theoretical framework is that machine classifiers perform not one but two incentivized tasks: choosing how to classify and learning how to classify. We show that while the adjustments engineers use correctly incentivize choosing, they can simultaneously reduce the incentives to learn. Our formal treatment of the problem reveals that methods embraced for their intuitive appeal can in fact misalign human and machine objectives in predictable ways.
Suggested Citation
David Autor & Andrew Caplin & Daniel J. Martin & Philip Marx, 2026.
"Misaligned by Design: incentive failures in machine learning,"
POID Working Papers
139, Centre for Economic Performance, LSE.
Handle:
RePEc:cep:poidwp:139
Download full text from publisher
Corrections
All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cep:poidwp:139. See general information about how to correct material in RePEc.
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
We have no bibliographic references for this item. You can help adding them by using this form .
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://poid.lse.ac.uk/_new/publications/working-papers/ .
Please note that corrections may take a couple of weeks to filter through
the various RePEc services.