Deep reinforcement learning-based energy scheduling for green buildings with stationary and EV batteries of heterogeneous characteristics

The substantial energy demands of buildings are increasingly supplied by renewable sources like photovoltaics. However, their intermittency necessitates the integration of stationary energy storage systems (ESS) within building energy management systems (BEMS) to stabilize power and coordinate multi-energy flows. The proliferation of electric vehicles (EVs) facilitates their integration with ESS, forming a combined battery system (CBS) that expands the arbitrage potential and flexibility of BEMS. To fully exploit the potential of CBS in optimizing BEMS operational costs, this paper proposes a deep reinforcement learning (DRL) real-time joint energy scheduling method based on heterogeneous battery systems. We first analyze the aging characteristics of different battery types within the CBS, and propose an innovative degradation assessment framework tailored to heterogeneous energy storage systems in vehicle-to-grid scenarios. This framework introduces a cycle degradation coefficient to provide real-time feedback on battery aging costs, making it suitable for DRL-driven scheduling. To achieve optimized collaborative scheduling of ESS and EVs, we propose an enhanced DRL algorithm incorporating double dueling and prioritized experience replay mechanisms. This algorithm addresses challenges such as complex state features, action coupling, and decreased learning efficiency in heterogeneous energy storage environments. It also prioritizes the travel demands of EV users to promote their participation. Experimental simulations from a real-world commercial building validate the effectiveness of the proposed approach, achieving a 43.39% reduction in system operating costs compared to the mixed-integer linear programming approach under equivalent conditions.