Simulating Extinction and its Role in Food Reinforcement Learning for Ultra-Processed Foods

Food environments such as supermarkets and school cafeterias are increasingly dominated by ultra-processed foods (UPFs). Consumption of these foods is partly shaped by reinforcement learning, where repeated intake leads to expected rewards. In contrast, encountering foods without consuming them can weaken these expectations through extinction — a process missing in existing models simulating food reinforcement learning. As a result, current models may fail to predict changes in learning, particularly following interventions like dieting or changes in food availability. We integrated extinction into a model of food reinforcement learning, allowing agents to update reward expectations through both consumption and non-consumption. First, we examine the conditions under which extinction prevents agents from learning to favor UPFs over less processed foods (LPFs) in environments with varying UPF availability. Second, we test interventions after agents learn in food environments resembling American supermarkets by varying (1) dieting probability and UPF availability, and (2) UPF reward value and availability. In prevention scenarios, extinction led agents to favor UPFs by suppressing learning opportunities for LPFs when UPFs were abundant. Learning outcomes were most sensitive to UPF availability. In intervention scenarios, learning shifted toward LPFs only when UPF availability was substantially reduced (from 75.9% to at most 21.6%), with minimal dieting required to initiate extinction. Neither dieting nor reducing UPF reward value alone was sufficient to reverse learning. Overall, both preventing and reversing learning for UPFs requires large reductions in their availability relative to current food environments.