A Class of Optimal Control Problems of Forward–Backward Systems with Input Constraint

In this paper, we consider a new class of optimal control problems with admissibility constraint, where the state is driven by a fully coupled forward backward stochastic differential equation (FBSDE) with mixed initial-terminal condition. Different from the classical control problems, both dynamic process control and static initial-terminal perturbations are considered. Moreover, all control/perturbation components are subject to input constraint in terms of closed convex sets and partial information in terms of some sub-filtration for randomness evolution. We first study the nonlinear case of aforementioned FBSDE optimal control by deriving stochastic maximum principle. Next, we consider the linear quadratic case with explicit representation of the optimal admissible controls. More specifically, a new Hamiltonian system involving three projection operators and conditional expectation is derived. Finally, we apply obtained maximum principle to study a general class of large-population system and provide a unified framework to analyze related mean-field game (MFG). Our result includes considerable existing MFG results as its special cases and provides some new features such as recursive functional or input delay average.