Adaptive NN–RL control for stochastic nonlinear systems with input and full state constraints

This paper investigates the tracking control problem of adaptive neural network (NN) reinforcement learning (RL) for the stochastic nonlinear systems with input and full state constraints. The proposed scheme combines a critic-action RL strategy with bounded control technology, ensuring that both the final controller and all states remain within the time-varying boundaries. In order to estimate the unmeasurable states, a state observer is constructed. Then, an adaptive NN-RL controller is designed. Different from the existing results, the proposed scheme consists of a basic controller and a compensation controller. For the basic controller, the NN approximation technique and the backstepping design method are adopted to address the tracking control problem. By utilising the error variables defined in the basic controller, the compensation controller is constructed based on critic-actor RL algorithm, which optimises the tracking performance while ensuring that the final controller adherence to the time-varying constraints. Due to the projection operator is introduced to design critic-action NN, the issue of gradient explosion encountered in gradient descent methods is averted. Finally, the stability of the closed-loop system is demonstrated via the tangent type time-varying barrier Lyapunov functions (BLFs). The simulation results verify the effectiveness of the proposed scheme.