Hierarchical fuzzy RL control strategy and economic evaluation for molten salt energy storage systems

Increasing integration of volatile renewable energy threatens grid security and raises demand for flexibility services such as deep peak shaving and rapid load adjustment from conventional coal-fired power plants (CFPPs). Energy storage systems (ESS) significantly enhance the load-adjustment capability of CFPPs, but the economic operation of thermal plants coupled with ESS is increasingly critical. In this study, a molten salt energy storage system (MSESS) is integrated with a CFPP to facilitate rapid and economical load adjustment. Two charging schemes are designed and analyzed for rapid and economical operation of the coupled system. To address limitations of traditional control strategies, a hierarchical control strategy based on multivariable fuzzy reinforcement learning is proposed. The strategy comprises an upper-level multivariable fuzzy control (MFC) to eliminate tracking errors in load adjustment and a lower-level reinforcement learning agent using a twin-delayed deep deterministic policy gradient (TD3) algorithm. The coupled system's flexibility demand is decomposed into a load-adjustment subproblem and an economic-operation subproblem, solved by the upper-level MFC and lower-level RL agent, respectively. A multi-criteria objective function is formulated, considering load-tracking cost, energy storage status, and coal consumption. A 48-h simulation under automatic power control (APC) regulation demonstrates that the proposed hierarchical RL-MFC strategy improves the load regulation performance of the coupled system by 1.05 % and enhances operational economy by 1.40 % compared to conventional strategies. Additionally, the proportion of the energy storage system operating within its optimal state-of-charge (SOC) range increased by 16.57 %