Blackwell Optimality In Stochastic Games

Blackwell optimality in a finite state-action discounted Markov decision process (MDP) gives an optimal strategy which is optimal for every discount factor close enough to one. In this article we explore this property, which we call as Blackwell–Nash equilibrium, in two player finite state-action discounted stochastic games. A strategy pair is said to be a Blackwell–Nash equilibrium if it is a Nash equilibrium for every discount factor close enough to one. A stationary Blackwell–Nash equilibrium in a stochastic game may not always exist as can be seen from "Big Match" example where a stationary Nash equilibrium does not exist in undiscounted case. For a Single Controller Additive Reward (SC-AR) stochastic game, we show that there exists a stationary deterministic Blackwell–Nash equilibrium which is also a Nash equilibrium for undiscounted case. For general stochastic games, we give some conditions which together are sufficient for any stationary Nash equilibrium of a discounted stochastic game to be a Blackwell–Nash equilibrium and it is also a Nash equilibrium of an undiscounted stochastic game. We illustrate our results on general stochastic games through a variant of the pollution tax model.