Shield-enhanced safe reinforcement learning control for wave energy converters

Ensuring safety in wave energy converter (WEC) systems against complex and changing ocean marine environments is essential for preventing structural damage, reducing maintenance costs, and enhancing long-term economic viability. Most WEC controllers capable of handling safety constraints are model-based, which face challenges such as modeling errors, nonlinear effects, and high real-time computational costs. Recently, model-free reinforcement learning (RL) techniques have received increasing attention due to their potential to learn control policies without relying on accurate models. Existing RL-based methods handle constraints by incorporating soft penalty terms into the reward function, but this approach has several limitations: it cannot ensure constraint satisfaction, may compromise energy capture efficiency, and requires costly tuning of penalty weights. To address these challenges, this paper proposes a novel shield-enhanced safe RL (SE-sRL) controller for WECs to enforce safety constraints while optimizing power extraction efficiency. In the shield mechanism, a WEC state prediction network recursively generates multi-step-ahead forecasts to detect potential constraint violations, based on which the control action is refined through a constraint-aware adjustment strategy. Several designs are proposed to enhance the efficacy and efficiency of the SE-sRL method, including shared layers for consistent feature extraction, two-stage training for dynamic-aware and constraint-compliant policy learning, and data augmentation for sustained prediction generalization. The effectiveness of the proposed SE-sRL controller is validated through simulations on a nonlinear WEC model, demonstrating improved efficiency while adhering to system constraints.