Deep reinforcement learning based electricity bill minimization strategy for residential prosumer

This paper presents a reinforcement learning (RL)-based optimization strategy for minimizing the electricity bill of a residential prosumer equipped with a photovoltaic (PV) system and a battery energy storage system (BESS). Specifically, we implement a Twin Delayed Deep Deterministic Policy Gradient (TD3) agent, an actor–critic RL algorithm with a continuous action space. The proposed model leverages historical PV generation and household consumption data to optimize the charging and discharging schedule of the BESS under a three-period tariff scheme. To enhance the learning process, a Long Short-Term Memory (LSTM) layer is integrated into both the actor and critic networks, allowing the agent to capture temporal dependencies in energy demand and PV generation. The RL agent is trained and evaluated using real-world data from a residential PV installation in Ireland, a region characterized by high variability of solar generation. The results demonstrate that the TD3-based approach effectively reduces electricity bill costs by strategically charging during low-tariff periods and discharging during peak-price hours, achieving an approximate 30% reduction in the monthly bill compared to a PV-only scenario, and a 21% relative to a benchmark strategy. In addition, the TD3 agent is compared to a Deep Deterministic Policy Gradient (DDPG) agent, another RL-based approach for continuous action spaces. The results confirm that TD3 outperforms DDPG in both cost savings and learning stability, validating the effectiveness of our proposed method.