Hidden Stochastic Games and Limit Equilibrium Payoffs

We consider 2-player stochastic games with perfectly observed actions, and study the limit, as the discount factor goes to one, of the equilibrium payoffs set. In the usual setup where current states are observed by the players, we first show that the set of stationary equilibrium payoffs always converges. We then provide the first examples where the whole set of equilibrium payoffs diverges. The construction can be robust to perturbations of the payoffs, and to the introduction of normal-form correlation. Next we naturally introduce the more general model of hidden stochastic game, where the players publicly receive imperfect signals over current states. In this setup we present a last example where not only the limit set of equilibrium payoffs does not exist, but there is no converging selection of equilibrium payoffs. The example is symmetric and robust in many aspects, in particular to the introduction of extensive-form correlation or communication devices. No uniform equilibrium payoff exists, and the equilibrium set has full support for each discount factor and each initial state.