Deep asynchronous gradient policy for cost-effective optimization of virtual energy hubs under uncertainty

—This paper presents a novel framework for virtual energy hub (VEH) optimization that integrates multiple energy carriers, including electrical, thermal, renewable energy units, and water. To meet diverse demands while considering energy markets signal. The proposed model contains conventional energy sources like combined heat and power units, gas boiler, and wind generation with more complex units like electrical vehicle parking lot (EVPL) and water desalination systems. In order to take into account uncertainties related to renewable generation and load variations, the proposed approach utilizes a demand response program for both electrical and thermal networks. In particular, deep asynchronous gradient policy (DAGP) has been adopted for solving the decision-making optimization problem by interacting with its agent with the environment. There are two deep neural networks (DNNs) in the architecture of DAGP to exploit the optimal policy for the VEH operation equipped with EVPL: i) an actor neural network generates optimal actions for allocation of energy units, and ii) a critic neural network is implemented to evaluate the quality of applied actions through estimating a pre-defined reinforcement signal. By training the capability of DNNs, the DAGP aims to facilitate energy trading in the electrical market while the agent responds to fluctuations in market price. Simulation tests on VEH under various scenarios reveal the feasibility of the suggested VEH optimization methodology (realized by the DAGP agent) to improve efficiency and reduce operational costs under uncertain situations.