Smart prosumers management based on multi-agent deep reinforcement learning to participate in decentralized peer-to-peer market

The integration of prosumers as sustainable and decentralized units has become a fundamental aspect of smart energy systems. It is therefore crucial to implement a market-oriented scheduling framework that guarantees the autonomous operation of prosumers while safeguarding their privacy, enhancing operational autonomy, and promoting competitiveness in electricity markets. In this regard, this paper puts forth a prosumer-centered approach to smart prosumer operations management, with the objective of enabling their participation in decentralized peer-to-peer (P2P) energy markets. To address this challenge, a multi-agent twin delayed deep deterministic policy gradient (MATD3) algorithm with a decentralized training and execution (DTDE) strategy is implemented, enabling prosumers to independently manage their demand, energy storage systems (ESS), and bidding/asking prices. This approach determines the demand of prosumers and the quantities of energy to be charged/discharged by their ESS in a dynamic manner, with the objective of making robust decisions regarding uncertainties arising from consumption and generation. Furthermore, a novel pricing mechanism comprising the mid-market rate (MMR) and MATD3 is proposed for decentralized P2P transactions within an adaptive continuous double auction (ACDA) market. Finally, a case study of 20 residential prosumers is presented, collectively engaged in P2P transactions to illustrate the efficacy of the proposed framework by achieving 9.8% and 13.2% scheduling errors, 5.2% P2P participation, and 10% revenue increment.