IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v15y2022i4p1440-d750803.html
   My bibliography  Save this article

Distributed Reinforcement Learning for the Management of a Smart Grid Interconnecting Independent Prosumers

Author

Listed:
  • Dominique Barth

    (DAVID Laboratory, UVSQ/Université Paris-Saclay, 45 Avenue des Etats Unis, 78035 Versailles, France
    These authors contributed equally to this work.)

  • Benjamin Cohen-Boulakia

    (LINEACT, CESI, 92000 Nanterre, France
    These authors contributed equally to this work.)

  • Wilfried Ehounou

    (LINEACT, CESI, 92000 Nanterre, France
    Laboratoire de Mathématiques Informatique, Université Nangui Abrogoua, Abidjan 02 BP V 102, Côte d’Ivoire
    These authors contributed equally to this work.)

Abstract

In the context of an eco-responsible production and distribution of electrical energy at the local scale of an urban territory, we consider a smart grid as a system interconnecting different prosumers, which all retain their decision-making autonomy and defend their own interests in a comprehensive system where the rules, accepted by all, encourage virtuous behavior. In this paper, we present and analyze a model and a management method for smart grids that is shared between different kinds of independent actors, who respect their own interests, and that encourages each actor to behavior that allows, as much as possible, an energy independence of the smart grid from external energy suppliers. We consider here a game theory model, in which each actor of the smart grid is a player, and we investigate distributed machine-learning algorithms to allow decision-making, thus, leading the game to converge to stable situations, in particular to a Nash equilibrium. We propose a Linear Reward Inaction algorithm that achieves Nash equilibria most of the time, both for a single time slot and across time, allowing the smart grid to maximize its energy independence from external energy suppliers.

Suggested Citation

  • Dominique Barth & Benjamin Cohen-Boulakia & Wilfried Ehounou, 2022. "Distributed Reinforcement Learning for the Management of a Smart Grid Interconnecting Independent Prosumers," Energies, MDPI, vol. 15(4), pages 1-19, February.
    • Handle: RePEc:gam:jeners:v:15:y:2022:i:4:p:1440-:d:750803
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/15/4/1440/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/15/4/1440/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Iver Bakken Sperstad & Magnus Korpås, 2019. "Energy Storage Scheduling in Distribution Systems Considering Wind and Photovoltaic Generation Uncertainties," Energies, MDPI, vol. 12(7), pages 1-24, March.
    2. Mohamadou Nassourou & Joaquim Blesa & Vicenç Puig, 2020. "Robust Economic Model Predictive Control Based on a Zonotope and Local Feedback Controller for Energy Dispatch in Smart-Grids Considering Demand Uncertainty," Energies, MDPI, vol. 13(3), pages 1-19, February.
    3. Chien, Steve & Sinclair, Alistair, 2011. "Convergence to approximate Nash equilibria in congestion games," Games and Economic Behavior, Elsevier, vol. 71(2), pages 315-327, March.
    4. Sylvain Cros & Jordi Badosa & André Szantaï & Martial Haeffelin, 2020. "Reliability Predictors for Solar Irradiance Satellite-Based Forecast," Energies, MDPI, vol. 13(21), pages 1-21, October.
    5. Milchtaich, Igal, 1996. "Congestion Games with Player-Specific Payoff Functions," Games and Economic Behavior, Elsevier, vol. 13(1), pages 111-124, March.
    6. Lu, Renzhi & Hong, Seung Ho, 2019. "Incentive-based demand response for smart grid with reinforcement learning and deep neural network," Applied Energy, Elsevier, vol. 236(C), pages 937-949.
    7. Calvillo, C.F. & Sánchez-Miralles, A. & Villar, J., 2016. "Energy management and planning in smart cities," Renewable and Sustainable Energy Reviews, Elsevier, vol. 55(C), pages 273-287.
    8. Ri Piao & Deok-Joo Lee & Taegu Kim, 2020. "Real-Time Pricing Scheme in Smart Grid Considering Time Preference: Game Theoretic Approach," Energies, MDPI, vol. 13(22), pages 1-19, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Omar Al-Ani & Sanjoy Das, 2022. "Reinforcement Learning: Theory and Applications in HEMS," Energies, MDPI, vol. 15(17), pages 1-37, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Maximilian Drees & Matthias Feldotto & Sören Riechers & Alexander Skopalik, 2019. "Pure Nash equilibria in restricted budget games," Journal of Combinatorial Optimization, Springer, vol. 37(2), pages 620-638, February.
    2. Muhammad Majid Hussain & Rizwan Akram & Zulfiqar Ali Memon & Mian Hammad Nazir & Waqas Javed & Muhammad Siddique, 2021. "Demand Side Management Techniques for Home Energy Management Systems for Smart Cities," Sustainability, MDPI, vol. 13(21), pages 1-20, October.
    3. Matthias Feldotto & Lennart Leder & Alexander Skopalik, 2018. "Congestion games with mixed objectives," Journal of Combinatorial Optimization, Springer, vol. 36(4), pages 1145-1167, November.
    4. Daskalakis, Constantinos & Papadimitriou, Christos H., 2015. "Approximate Nash equilibria in anonymous games," Journal of Economic Theory, Elsevier, vol. 156(C), pages 207-245.
    5. Biancardi, Marta & Di Bari, Antonio & Villani, Giovanni, 2021. "R&D investment decision on smart cities: Energy sustainability and opportunity," Chaos, Solitons & Fractals, Elsevier, vol. 153(P2).
    6. Zhang, Yang & Yang, Qingyu & Li, Donghe & An, Dou, 2022. "A reinforcement and imitation learning method for pricing strategy of electricity retailer with customers’ flexibility," Applied Energy, Elsevier, vol. 323(C).
    7. Arnold, Tone & Wooders, Myrna, 2002. "Dynamic Club Formation with Coordination," Economic Research Papers 269414, University of Warwick - Department of Economics.
    8. Hideo Konishi, 2004. "Uniqueness of User Equilibrium in Transportation Networks with Heterogeneous Commuters," Transportation Science, INFORMS, vol. 38(3), pages 315-330, August.
    9. Ibrahim, Muhammad Sohail & Dong, Wei & Yang, Qiang, 2020. "Machine learning driven smart electric power systems: Current trends and new perspectives," Applied Energy, Elsevier, vol. 272(C).
    10. Attour, Amel & Baudino, Marco & Krafft, Jackie & Lazaric, Nathalie, 2020. "Determinants of energy tracking application use at the city level: Evidence from France," Energy Policy, Elsevier, vol. 147(C).
    11. Milchtaich, Igal, 2009. "Weighted congestion games with separable preferences," Games and Economic Behavior, Elsevier, vol. 67(2), pages 750-757, November.
    12. Davarzani, Sima & Pisica, Ioana & Taylor, Gareth A. & Munisami, Kevin J., 2021. "Residential Demand Response Strategies and Applications in Active Distribution Network Management," Renewable and Sustainable Energy Reviews, Elsevier, vol. 138(C).
    13. Vo-Van Thanh & Wencong Su & Bin Wang, 2022. "Optimal DC Microgrid Operation with Model Predictive Control-Based Voltage-Dependent Demand Response and Optimal Battery Dispatch," Energies, MDPI, vol. 15(6), pages 1-19, March.
    14. Bavly, Gilad & Heller, Yuval & Schreiber, Amnon, 2022. "Social welfare in search games with asymmetric information," Journal of Economic Theory, Elsevier, vol. 202(C).
    15. Nikolaos Efkarpidis & Andrija Goranović & Chen-Wei Yang & Martin Geidl & Ingo Herbst & Stefan Wilker & Thilo Sauter, 2022. "A Generic Framework for the Definition of Key Performance Indicators for Smart Energy Systems at Different Scales," Energies, MDPI, vol. 15(4), pages 1-30, February.
    16. Balta, Münevver Özge & Balta, Mustafa Tolga, 2022. "Development of a sustainable hydrogen city concept and initial hydrogen city projects," Energy Policy, Elsevier, vol. 166(C).
    17. Xu, Fangyuan & Zhu, Weidong & Wang, Yi Fei & Lai, Chun Sing & Yuan, Haoliang & Zhao, Yujia & Guo, Siming & Fu, Zhengxin, 2022. "A new deregulated demand response scheme for load over-shifting city in regulated power market," Applied Energy, Elsevier, vol. 311(C).
    18. Darryl Seale & Amnon Rapoport, 2000. "Elicitation of Strategy Profiles in Large Group Coordination Games," Experimental Economics, Springer;Economic Science Association, vol. 3(2), pages 153-179, October.
    19. Kalim Ullah & Sajjad Ali & Taimoor Ahmad Khan & Imran Khan & Sadaqat Jan & Ibrar Ali Shah & Ghulam Hafeez, 2020. "An Optimal Energy Optimization Strategy for Smart Grid Integrated with Renewable Energy Sources and Demand Response Programs," Energies, MDPI, vol. 13(21), pages 1-17, November.
    20. Pinto, Giuseppe & Deltetto, Davide & Capozzoli, Alfonso, 2021. "Data-driven district energy management with surrogate models and deep reinforcement learning," Applied Energy, Elsevier, vol. 304(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:15:y:2022:i:4:p:1440-:d:750803. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.