Semi-Markov control models for systems of large populations of interacting objects with possible unbounded costs: a mean field approach

This paper is about optimal control problems associated to stochastic systems composed of a large number of N ( $$N\sim \infty $$ N ∼ ∞ ) interacting objects (e.g., particles, agents, data, etc.) evolving among a finite or countable set of classes or categories according to a semi-Markov process. Such systems are modeled by a control model $$\mathcal{S}\mathcal{M}_{N}$$ S M N where the states are vectors whose components are the proportions of objects in each class. Since N is too large, from a practical point of view, it is almost impossible to obtain a solution of the control problem. Under this setting, we apply a mean field approach which consists of letting $$N\rightarrow \infty $$ N → ∞ (the mean field limit). Then we obtain the mean field control model $$\mathcal{S}\mathcal{M}$$ S M , independent on N, which is easier to study than $$\mathcal{S}\mathcal{M}_{N}.$$ S M N . Our main objective is to show that an optimal policy $$\pi _{*},$$ π ∗ , under a discounted criterion, in $$\mathcal{S}\mathcal{M}$$ S M has a good behavior in $$\mathcal{S}\mathcal{M}_{N}.$$ S M N . Specifically, we prove that $$\pi _{*}$$ π ∗ is nearly discounted optimal in $$\mathcal{S}\mathcal{M}_{N}$$ S M N asymptotically as $$N\rightarrow \infty .$$ N → ∞ .