N-player Pareto game for the discrete-time Markov jump systems with multiplicative noises using off-policy reinforcement learning

This paper investigates a discrete-time N-player Pareto cooperative game problem for the Markov jump systems with multiplicative noises. First, utilizing the convexity condition, the Pareto cooperative game problem is transformed into a weighted sum optimal control problem. To address this problem, we reconstruct the system and prove that the optimal control gains do not change after the reconstruction. Compared with the traditional policy iteration algorithm which is sensitive to probing noise, the proposed model-based off-policy algorithm yields unbiased solution to Pareto cooperative game problem when probing noise is added. Due to the complexity of the jump systems and the multiplicative noises, relying on its dynamic information may cause policy bias in some situations. To further eliminate the dependence on the system information of model-based off-policy algorithm, a model-free off-policy algorithm is designed based on the properties of the trace using only information of the system states and inputs. In addition, the convergence of the proposed algorithm and the stability of the system under each iteration step are proved. Finally, practical numerical examples are given to show the effectiveness of the algorithm.