A numerical optimization pesudo-algorithm for two-player zero-sum stochastic games

A two-player, zero-sum stochastic game with two variables $$(z,v)$$(z,v) can be transformed into the coupled control PDEs by a transition probability matrix of the Markov chain. There exist two control variables $${\varsigma _i}$$ςi and two tensity rates $${\mu _i}$$μi ($$i = 1,2$$i=1,2) after the transformation that should be solved for. Yuan and Li (Computational Economics, 2018) give a numerical algorithm by the first-order necessary condition, which overcomes several drawbacks of the normal algorithm and describe several numerical experiments demonstrating the performance of their method. However, their method exhibits at least one shortcoming, which is that the final value of the control variables $${\varsigma _i}$$ςi ($$i = 1,2$$i=1,2) may exceed the definition domain. This situation means that the $${\varsigma _i}$$ςi ($$i = 1,2$$i=1,2) is infeasible and unacceptable. This paper presents a new technique to avoid that situation, and an optimization pseudo-algorithm is designed using the following steps: (i) starting from the given initial points $$(\varsigma _i^0,\mu _i^0),$$(ςi0,μi0), an active-set algorithm is proposed; (ii) the limited memory update technique is used in the algorithm to obtain fast convergence and low storage; (iii) global convergence is established under suitable conditions; and (iv) numerical results are reported to demonstrate that the new algorithm is competitive with the normal algorithm.