Reinforcement Learning for Integrated MPPT and Battery Management in Photovoltaic Systems: A Systematic Review

This work provides a systematic literature review on reinforcement learning (RL) for integrated maximum power point tracking (MPPT) and battery management in photovoltaic (PV) systems. As PV installations increasingly incorporate battery energy storage, the traditional objective of maximizing instantaneous power extraction is no longer sufficient on its own, since control decisions also affect battery state of charge, efficiency, degradation, and load support. Although RL has shown strong potential for sequential decision making in energy systems, most existing studies still treat MPPT and battery management as separate or only loosely coordinated problems. This review examines this issue by systematically examining how RL, particularly continuous-action methods, has been applied to coupled PV–battery control. The analysis highlights the shortcomings of discrete-action formulations in power-electronic systems and emphasizes the advantages and limitations of actor–critic approaches such as DDPG, TD3, PPO, and SAC for directly optimizing continuous-control variables. Across the reviewed literature, RL is found to be used predominantly at the supervisory energy management level, with PV generation often treated as exogenous rather than as an explicit control decision. This review therefore identifies a persistent structural separation between converter-level PV control and storage-aware energy management within a common learning and evaluation framework. It identifies continuous-action RL as a candidate formulation for unified PV–battery optimization while highlighting important challenges in constraint handling, state representation, sample efficiency, stability, and hardware validation.