Simulating Extinction and its Role in Food Reinforcement Learning for Ultra-Processed Foods

Food environments, the settings where people encounter food and make choices, are increasingly dominated by ultra-processed foods. Consumption of these foods is partly driven by food reinforcement learning, where repeated consumption leads to expected rewards. Conversely, repeated encounters with foods without consumption may weaken expected rewards, a mechanism called extinction. Interventions such as dieting or altering the food environment may implicitly rely on extinction by creating encounters with food without consumption; however, extinction is missing in computational models of food reinforcement learning. Without extinction, models may fail to predict changes in food reinforcement learning, particularly following interventions. Hence, we integrated extinction into a computational model of food reinforcement learning, in which agents traverse food environments and learn rewards through food consumption. First, we simulated how extinction shapes food reinforcement learning for agents with no prior learning experiences in various food environments, using food reward values informed by neuroscientific evidence. Next, we simulated agents learning in a food environment parameterized using a database of over 50,000 products in American supermarkets, after which we intervened by changing the degree of dieting or the availability of foods in the food environment. Last, we examined conditions leading to irreversibility, a state in which learned rewards for less processed foods can no longer overtake those of ultra-processed foods. Including extinction increased learning for ultra-processed foods by suppressing learning opportunities for less processed foods. While reducing the relative availability of ultra-processed foods produced larger shifts in learning and consumption than dieting, a small amount of dieting was necessary to counteract previously learned rewards. Larger differences between food reward values produced stronger learning toward ultra-processed foods and increased irreversibility. These findings show how extinction alters learning and how current food environments shape learning in ways that may be difficult to reverse.