On the Existence of Markov Perfect Equilibria in Perfect Information Games

We study the existence of pure strategy Markov perfect equilibria in two-person perfect information games. There is a state space X and each period player's possible actions are a subset of X. This set of feasible actions depends on the current state, which is determined by the choice of the other player in the previous period. We assume that X is a compact Hausdorff space and that the action correspondence has an acyclic and asymmetric graph. For some states there may be no feasible actions and then the game ends. Payoffs are either discounted sums of utilities of the states visited, or the utility of the state where the game ends. We give sufficient conditions for the existence of equilibrium e.g. in case when either feasible action sets are finite or when players' payoffs are continuously dependent on each other. The latter class of games includes zero-sum games and pure coordination games.