IDEAS home Printed from https://ideas.repec.org/a/eee/energy/v59y2013icp133-146.html
   My bibliography  Save this article

Reinforcement learning for microgrid energy management

Author

Listed:
  • Kuznetsova, Elizaveta
  • Li, Yan-Fu
  • Ruiz, Carlos
  • Zio, Enrico
  • Ault, Graham
  • Bell, Keith

Abstract

We consider a microgrid for energy distribution, with a local consumer, a renewable generator (wind turbine) and a storage facility (battery), connected to the external grid via a transformer. We propose a 2 steps-ahead reinforcement learning algorithm to plan the battery scheduling, which plays a key role in the achievement of the consumer goals. The underlying framework is one of multi-criteria decision-making by an individual consumer who has the goals of increasing the utilization rate of the battery during high electricity demand (so as to decrease the electricity purchase from the external grid) and increasing the utilization rate of the wind turbine for local use (so as to increase the consumer independence from the external grid). Predictions of available wind power feed the reinforcement learning algorithm for selecting the optimal battery scheduling actions. The embedded learning mechanism allows to enhance the consumer knowledge about the optimal actions for battery scheduling under different time-dependent environmental conditions. The developed framework gives the capability to intelligent consumers to learn the stochastic environment and make use of the experience to select optimal energy management actions.

Suggested Citation

  • Kuznetsova, Elizaveta & Li, Yan-Fu & Ruiz, Carlos & Zio, Enrico & Ault, Graham & Bell, Keith, 2013. "Reinforcement learning for microgrid energy management," Energy, Elsevier, vol. 59(C), pages 133-146.
  • Handle: RePEc:eee:energy:v:59:y:2013:i:c:p:133-146
    DOI: 10.1016/j.energy.2013.05.060
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0360544213004817
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.energy.2013.05.060?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Niknam, Taher & Azizipanah-Abarghooee, Rasoul & Narimani, Mohammad Rasoul, 2012. "An efficient scenario-based stochastic programming framework for multi-objective optimal micro-grid operation," Applied Energy, Elsevier, vol. 99(C), pages 455-470.
    2. Jun, Zeng & Junfeng, Liu & Jie, Wu & Ngan, H.W., 2011. "A multi-agent solution to energy management in hybrid renewable energy generation system," Renewable Energy, Elsevier, vol. 36(5), pages 1352-1363.
    3. Hledik, Ryan, 2009. "How Green Is the Smart Grid?," The Electricity Journal, Elsevier, vol. 22(3), pages 29-41, April.
    4. Li, Yan-Fu & Zio, Enrico, 2012. "A multi-state model for the reliability assessment of a distributed generation system via universal generating function," Reliability Engineering and System Safety, Elsevier, vol. 106(C), pages 28-36.
    5. Niknam, Taher & Golestaneh, Faranak & Malekpour, Ahmadreza, 2012. "Probabilistic energy and operation management of a microgrid containing wind/photovoltaic/fuel cell generation and energy storage devices based on point estimate method and self-adaptive gravitational," Energy, Elsevier, vol. 43(1), pages 427-437.
    6. Shamshad, A. & Bawadi, M.A. & Wan Hussin, W.M.A. & Majid, T.A. & Sanusi, S.A.M., 2005. "First and second order Markov chain models for synthetic generation of wind speed time series," Energy, Elsevier, vol. 30(5), pages 693-708.
    7. Ma, Tieju & Nakamori, Yoshiteru, 2009. "Modeling technological change in energy systems – From optimization to agent-based modeling," Energy, Elsevier, vol. 34(7), pages 873-879.
    8. Louit, D.M. & Pascual, R. & Jardine, A.K.S., 2009. "A practical procedure for the selection of time-to-failure models based on the assessment of trends in maintenance data," Reliability Engineering and System Safety, Elsevier, vol. 94(10), pages 1618-1628.
    9. Weidlich, Anke & Veit, Daniel, 2008. "A critical survey of agent-based wholesale electricity market models," Energy Economics, Elsevier, vol. 30(4), pages 1728-1759, July.
    10. Yousefi, Shaghayegh & Moghaddam, Mohsen Parsa & Majd, Vahid Johari, 2011. "Optimal real time pricing in an agent-based retail market using a comprehensive demand response model," Energy, Elsevier, vol. 36(9), pages 5716-5727.
    11. Manfren, Massimiliano & Caputo, Paola & Costa, Gaia, 2011. "Paradigm shift in urban energy systems through distributed generation: Methods and models," Applied Energy, Elsevier, vol. 88(4), pages 1032-1048, April.
    12. Mahvi, M. & Ardehali, M.M., 2011. "Optimal bidding strategy in a competitive electricity market based on agent-based approach and numerical sensitivity analysis," Energy, Elsevier, vol. 36(11), pages 6367-6374.
    13. Dalili, N. & Edrisy, A. & Carriveau, R., 2009. "A review of surface engineering issues critical to wind turbine performance," Renewable and Sustainable Energy Reviews, Elsevier, vol. 13(2), pages 428-438, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kuznetsova, Elizaveta & Li, Yan-Fu & Ruiz, Carlos & Zio, Enrico, 2014. "An integrated framework of agent-based modelling and robust optimization for microgrid energy management," Applied Energy, Elsevier, vol. 129(C), pages 70-88.
    2. Chiacchio, Ferdinando & D’Urso, Diego & Famoso, Fabio & Brusca, Sebastian & Aizpurua, Jose Ignacio & Catterson, Victoria M., 2018. "On the use of dynamic reliability for an accurate modelling of renewable power plants," Energy, Elsevier, vol. 151(C), pages 605-621.
    3. Ahmad Khan, Aftab & Naeem, Muhammad & Iqbal, Muhammad & Qaisar, Saad & Anpalagan, Alagan, 2016. "A compendium of optimization objectives, constraints, tools and algorithms for energy management in microgrids," Renewable and Sustainable Energy Reviews, Elsevier, vol. 58(C), pages 1664-1683.
    4. Izadbakhsh, Maziar & Gandomkar, Majid & Rezvani, Alireza & Ahmadi, Abdollah, 2015. "Short-term resource scheduling of a renewable energy based micro grid," Renewable Energy, Elsevier, vol. 75(C), pages 598-606.
    5. Velik, Rosemarie & Nicolay, Pascal, 2014. "Grid-price-dependent energy management in microgrids using a modified simulated annealing triple-optimizer," Applied Energy, Elsevier, vol. 130(C), pages 384-395.
    6. Roy, Kallol & Mandal, Kamal Krishna & Mandal, Atis Chandra, 2019. "Ant-Lion Optimizer algorithm and recurrent neural network for energy management of micro grid connected system," Energy, Elsevier, vol. 167(C), pages 402-416.
    7. Sharma, Sharmistha & Bhattacharjee, Subhadeep & Bhattacharya, Aniruddha, 2018. "Probabilistic operation cost minimization of Micro-Grid," Energy, Elsevier, vol. 148(C), pages 1116-1139.
    8. Fazlalipour, Pary & Ehsan, Mehdi & Mohammadi-Ivatloo, Behnam, 2019. "Risk-aware stochastic bidding strategy of renewable micro-grids in day-ahead and real-time markets," Energy, Elsevier, vol. 171(C), pages 689-700.
    9. Shin, Joohyun & Lee, Jay H. & Realff, Matthew J., 2017. "Operational planning and optimal sizing of microgrid considering multi-scale wind uncertainty," Applied Energy, Elsevier, vol. 195(C), pages 616-633.
    10. Mavromatidis, Georgios & Orehounig, Kristina & Carmeliet, Jan, 2018. "A review of uncertainty characterisation approaches for the optimal design of distributed energy systems," Renewable and Sustainable Energy Reviews, Elsevier, vol. 88(C), pages 258-277.
    11. Kraan, O. & Kramer, G.J. & Nikolic, I., 2018. "Investment in the future electricity system - An agent-based modelling approach," Energy, Elsevier, vol. 151(C), pages 569-580.
    12. Saeian, Hosein & Niknam, Taher & Zare, Mohsen & Aghaei, Jamshid, 2022. "Coordinated optimal bidding strategies methods of aggregated microgrids: A game theory-based demand side management under an electricity market environment," Energy, Elsevier, vol. 245(C).
    13. Koppelaar, Rembrandt H.E.M. & Keirstead, James & Shah, Nilay & Woods, Jeremy, 2016. "A review of policy analysis purpose and capabilities of electricity system models," Renewable and Sustainable Energy Reviews, Elsevier, vol. 59(C), pages 1531-1544.
    14. Jin, Ming & Feng, Wei & Marnay, Chris & Spanos, Costas, 2018. "Microgrid to enable optimal distributed energy retail and end-user demand response," Applied Energy, Elsevier, vol. 210(C), pages 1321-1335.
    15. Bahmani-Firouzi, Bahman & Farjah, Ebrahim & Azizipanah-Abarghooee, Rasoul, 2013. "An efficient scenario-based and fuzzy self-adaptive learning particle swarm optimization approach for dynamic economic emission dispatch considering load and wind power uncertainties," Energy, Elsevier, vol. 50(C), pages 232-244.
    16. Gamarra, Carlos & Guerrero, Josep M., 2015. "Computational optimization techniques applied to microgrids planning: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 48(C), pages 413-424.
    17. Danny Espín-Sarzosa & Rodrigo Palma-Behnke & Oscar Núñez-Mata, 2020. "Energy Management Systems for Microgrids: Main Existing Trends in Centralized Control Architectures," Energies, MDPI, vol. 13(3), pages 1-32, January.
    18. Mo, Hua-Dong & Li, Yan-Fu & Zio, Enrico, 2016. "A system-of-systems framework for the reliability analysis of distributed generation systems accounting for the impact of degraded communication networks," Applied Energy, Elsevier, vol. 183(C), pages 805-822.
    19. Auke Hoekstra & Maarten Steinbuch & Geert Verbong, 2017. "Creating Agent-Based Energy Transition Management Models That Can Uncover Profitable Pathways to Climate Change Mitigation," Complexity, Hindawi, vol. 2017, pages 1-23, December.
    20. Haghnevis, Moeed & Askin, Ronald G. & Armbruster, Dieter, 2016. "An agent-based modeling optimization approach for understanding behavior of engineered complex adaptive systems," Socio-Economic Planning Sciences, Elsevier, vol. 56(C), pages 67-87.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:energy:v:59:y:2013:i:c:p:133-146. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/energy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.