Hidden stochastic games and limit equilibrium payoffs

We introduce the model of hidden stochastic games, which are stochastic games where players observe past actions and public signals on the current state. The natural state variable for these games is the common belief over the current state of the stochastic game. In this setup, we present an example in which the limit set of equilibrium payoffs, as the discount factor goes to 1, does not exist. Although the equilibrium payoff sets have full dimension, there is no converging selection of equilibrium payoffs. The example is symmetric and robust in many aspects, and in particular to extensive-form correlation or communication devices. No reasonable limit equilibrium payoff exists, and it is difficult to give any good answer to the question: “In the game played by extremely patient players, what are the possible outcomes?” The construction generalizes on a recent zero-sum example (Ziliotto, 2016), while improving significantly its properties.