Critical Dynamics in the Evolution of Stochastic Strategies for the Iterated Prisoner's Dilemma

The observed cooperation on the level of genes, cells, tissues, and individuals has been the object of intense study by evolutionary biologists, mainly because cooperation often flourishes in biological systems in apparent contradiction to the selfish goal of survival inherent in Darwinian evolution. In order to resolve this paradox, evolutionary game theory has focused on the Prisoner's Dilemma (PD), which incorporates the essence of this conflict. Here, we encode strategies for the iterated Prisoner's Dilemma (IPD) in terms of conditional probabilities that represent the response of decision pathways given previous plays. We find that if these stochastic strategies are encoded as genes that undergo Darwinian evolution, the environmental conditions that the strategies are adapting to determine the fixed point of the evolutionary trajectory, which could be either cooperation or defection. A transition between cooperative and defective attractors occurs as a function of different parameters such as mutation rate, replacement rate, and memory, all of which affect a player's ability to predict an opponent's behavior. These results imply that in populations of players that can use previous decisions to plan future ones, cooperation depends critically on whether the players can rely on facing the same strategies that they have adapted to. Defection, on the other hand, is the optimal adaptive response in environments that change so quickly that the information gathered from previous plays cannot usefully be integrated for a response.Author Summary: The observed cooperation between genes, cells, tissues, and higher organisms represents a paradox for Darwinian evolution, because the individual success of cheating is rewarded before its long-term detrimental consequences are felt. The tension between cooperation and defection can be represented by a simple game (the “Prisoner's Dilemma”), which has been used to study the conflicts between decisions to cooperate or defect. Here, we encode these decisions within genes, and allow them to adapt to environments that differ in how well a player can predict how an opponent is going to play. We find that evolutionary paths end at strategies that cooperate if the environment is sufficiently predictable, while they end in defection in uncertain and inconsistent worlds because inconsistency favors defection over cooperation. This work shows that cooperation or defection, in populations of players that use the information from previous moves to plan future ones, can be influenced by changing the environmental parameters.