From linear to nonlinear: Deriving long-term operating rules for hydro-wind-photovoltaic complementary systems with deep reinforcement learning

Hydro-wind-photovoltaic (PV) complementary systems (HWPCSs) provide an effective pathway for integrating variable wind and PV power. In such systems, well-designed long-term operating rules are essential for the efficient utilization of renewable resources. However, commonly used linear operating rules (LORs) oversimplify the complex relationships between optimal operational decisions and various input factors, thereby limiting the realization of HWPCSs' full potential. To overcome these limitations, this study introduces nonlinear operating rules by proposing a deep reinforcement learning (DRL) framework. First, a long-term operation model is constructed to identify the key elements in the DRL framework (e.g., action, state, and reward). Second, the proximal policy optimization algorithm is employed to derive the nonlinear operating rules, with action projection incorporated to ensure the satisfaction of all physical and operational constraints. Results in a case study indicate that the DRL method effectively derives stable operating rules while satisfying all constraints. Compared with the LORs, the DRL-based operating rules increase the average annual energy generation of the HWPCS by 0.38 billion kWh (+1.65%) and reduce the curtailment rate by 3.76%. Overall, the DRL method can drive operational decisions closer to the theoretical optimum by better capturing the nonlinear relationships between input factors and optimal decisions. The proposed DRL-based nonlinear operating rules can provide valuable guidelines for the effective operation of HWPCSs.