Nash equilibrium of attack and defense behaviors between predators and prey

How animals process information, compute, and execute behaviors is a central question in neuroscience and computational biology. Predators attack prey by chasing or ambushing them, while prey respond with escaping or freezing. These behaviors are fundamental for survival. Uncovering functions of such behaviors requires an understanding not only of the implementation of neuronal circuits but also of the underlying algorithms and computation. However, how animals respond to predators or prey depending on whether they can detect them from a distance remains unclear. Here, we modeled and analyzed attack and defense behaviors with game theory. Using encounter probabilities to construct payoff matrices under a sensory–motor algorithm that lacked directional information, we identified the corresponding equilibrium behaviors for the agents (predators and prey). Different detection distances yielded distinct Nash equilibrium behaviors, representing a computational mechanism that can account for diverse attack and defense behaviors. The games based on interactions among multiple predators and prey were, in most cases, non-constant-sum and positive-sum games. Measured payoffs of Nash equilibrium behaviors indicated that the predators were able to increase their payoffs by attacking, and the prey were also able to increase their payoffs even in the presence of predators. These results suggest that each of the agents initiates attack and defense behaviors. Moreover, Nash equilibrium behaviors were also identified under a simpler non-sensory motor algorithm. Despite the similarity, the non-sensory motor algorithm and the sensory–motor algorithm had distinct adaptive significance. The sensory–motor algorithm produced substantially greater prey payoffs. By implementing these algorithms, agents interact in ways that give rise to payoff matrices from which various Nash equilibrium behaviors can be mathematically derived under different conditions. Furthermore, this approach offers an experimental framework for understanding behavioral evolution and suggests a possible difference in evolutionary mechanisms of attack and defense behaviors.Author summary: Predators attack prey by chasing or ambushing, while prey respond to predator threats with defense behaviors such as escaping or freezing. These behaviors are fundamental survival strategies for predators and prey (agents). However, how animals respond to predators or prey depending on whether they can detect them from a distance remains unclear. Using individual-based models, we simulated and analyzed predator–prey interactions and computed the Nash equilibria of the resulting payoff matrices; at these equilibria, no agent can increase its payoff by a unilateral deviation. Different detection distances resulted in distinct Nash equilibrium behaviors, including chasing and ambushing by predators, escaping and freezing by prey, and their combinations. Furthermore, predators can increase their payoffs by attacking, and prey can also increase their payoffs even in the presence of predators, suggesting that attack and defense may be initiated automatically. The results indicate a computational mechanism that generates various attack and defense behaviors. Finally, we suggest that attack and defense behaviors may follow different evolutionary mechanisms. This is due to asymmetries in predator–prey interactions because predators can learn through trial and error, whereas prey cannot learn from fatal defense failures.