A Transformer-Based Deep Reinforcement Learning Method for Controller Parameter Modulation in Fault-Tolerant Control

This paper proposes a Transformer-based deep reinforcement learning method for adaptive controller parameter modulation. Unlike conventional approaches relying on metaheuristic optimization with fault-specific tuning or model-based gain scheduling, the proposed method learns a unified parameter modulation policy through direct environment interaction without requiring pre-computed optimal solutions. The key innovation lies in a parameter tokenization mechanism that represents each controller parameter as an independent token, enabling self-attention to capture cross-parameter dependencies for coordinated adaptation. A sequential state encoder extracts temporal fault evolution patterns, while fault-aware cross-attention integrates fault context to guide parameter adjustment according to varying fault types and severities. The policy is trained end-to-end using Proximal Policy Optimization with randomized fault injection. Experiments across three systems demonstrate consistent improvements: compared with GA-based tuning, the proposed method achieves lower ISE using a single policy without fault-specific re-optimization; against PSO-based backstepping control, the proposed method achieves tighter error bounds; compared with TD3-based PI scheduling, RMSE is reduced by 55% and recovery time by 47% under time-varying faults. These results validate that the proposed architecture enables effective fault-aware parameter modulation while preserving baseline controller structure.