IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v11y2023i9p2166-d1139635.html
   My bibliography  Save this article

Maximum Power Point Tracker Controller for Solar Photovoltaic Based on Reinforcement Learning Agent with a Digital Twin

Author

Listed:
  • Eneko Artetxe

    (Department Systems Engineering and Automatic Control, Faculty of Engineering of Vitoria-Gasteiz, University of the Basque Country (UPV/EHU), 01006 Vitoria-Gasteiz, Spain)

  • Jokin Uralde

    (Department Systems Engineering and Automatic Control, Faculty of Engineering of Vitoria-Gasteiz, University of the Basque Country (UPV/EHU), 01006 Vitoria-Gasteiz, Spain)

  • Oscar Barambones

    (Department Systems Engineering and Automatic Control, Faculty of Engineering of Vitoria-Gasteiz, University of the Basque Country (UPV/EHU), 01006 Vitoria-Gasteiz, Spain)

  • Isidro Calvo

    (Department Systems Engineering and Automatic Control, Faculty of Engineering of Vitoria-Gasteiz, University of the Basque Country (UPV/EHU), 01006 Vitoria-Gasteiz, Spain)

  • Imanol Martin

    (Department Systems Engineering and Automatic Control, Faculty of Engineering of Vitoria-Gasteiz, University of the Basque Country (UPV/EHU), 01006 Vitoria-Gasteiz, Spain)

Abstract

Photovoltaic (PV) energy, representing a renewable source of energy, plays a key role in the reduction of greenhouse gas emissions and the achievement of a sustainable mix of energy generation. To achieve the maximum solar energy harvest, PV power systems require the implementation of Maximum Power Point Tracking (MPPT). Traditional MPPT controllers, such as P&O, are easy to implement, but they are by nature slow and oscillate around the MPP losing efficiency. This work presents a Reinforcement learning (RL)-based control to increase the speed and the efficiency of the controller. Deep Deterministic Policy Gradient (DDPG), the selected RL algorithm, works with continuous actions and space state to achieve a stable output at MPP. A Digital Twin (DT) enables simulation training, which accelerates the process and allows it to operate independent of weather conditions. In addition, we use the maximum power achieved in the DT to adjust the reward function, making the training more efficient. The RL control is compared with a traditional P&O controller to validate the speed and efficiency increase both in simulations and real implementations. The results show an improvement of 10.45% in total power output and a settling time 24.54 times faster in simulations. Moreover, in real-time tests, an improvement of 51.45% in total power output and a 0.25 s settling time of the DDPG compared with 4.26 s of the P&O is obtained.

Suggested Citation

  • Eneko Artetxe & Jokin Uralde & Oscar Barambones & Isidro Calvo & Imanol Martin, 2023. "Maximum Power Point Tracker Controller for Solar Photovoltaic Based on Reinforcement Learning Agent with a Digital Twin," Mathematics, MDPI, vol. 11(9), pages 1-21, May.
    • Handle: RePEc:gam:jmathe:v:11:y:2023:i:9:p:2166-:d:1139635
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/11/9/2166/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/11/9/2166/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Chendi Li & Yuanrui Chen & Dongbao Zhou & Junfeng Liu & Jun Zeng, 2016. "A High-Performance Adaptive Incremental Conductance MPPT Algorithm for Photovoltaic Systems," Energies, MDPI, vol. 9(4), pages 1-17, April.
    2. Maissa Farhat & Oscar Barambones & Lassaâd Sbita, 2020. "A Real-Time Implementation of Novel and Stable Variable Step Size MPPT," Energies, MDPI, vol. 13(18), pages 1-18, September.
    3. John Macaulay & Zhongfu Zhou, 2018. "A Fuzzy Logical-Based Variable Step Size P&O MPPT Algorithm for Photovoltaic System," Energies, MDPI, vol. 11(6), pages 1-15, May.
    4. Fathi Troudi & Houda Jouini & Abdelkader Mami & Nidhal Ben Khedher & Walid Aich & Attia Boudjemline & Mohamed Boujelbene, 2022. "Comparative Assessment between Five Control Techniques to Optimize the Maximum Power Point Tracking Procedure for PV Systems," Mathematics, MDPI, vol. 10(7), pages 1-15, March.
    5. Kofinas, P. & Doltsinis, S. & Dounis, A.I. & Vouros, G.A., 2017. "A reinforcement learning approach for MPPT control method of photovoltaic sources," Renewable Energy, Elsevier, vol. 108(C), pages 461-473.
    6. Efrain Mendez & Alexandro Ortiz & Pedro Ponce & Israel Macias & David Balderas & Arturo Molina, 2020. "Improved MPPT Algorithm for Photovoltaic Systems Based on the Earthquake Optimization Algorithm," Energies, MDPI, vol. 13(12), pages 1-24, June.
    7. Marcel Nicola & Claudiu-Ionel Nicola & Dan Selișteanu, 2022. "Improvement of the Control of a Grid Connected Photovoltaic System Based on Synergetic and Sliding Mode Controllers Using a Reinforcement Learning Deep Deterministic Policy Gradient Agent," Energies, MDPI, vol. 15(7), pages 1-32, March.
    8. Mehmet Ali Yildirim & Marzena Nowak-Ocłoń, 2020. "Modified Maximum Power Point Tracking Algorithm under Time-Varying Solar Irradiation," Energies, MDPI, vol. 13(24), pages 1-15, December.
    9. Safyan Mukhtar & Taza Gul, 2023. "Solar Radiation and Thermal Convection of Hybrid Nanofluids for the Optimization of Solar Collector," Mathematics, MDPI, vol. 11(5), pages 1-13, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mitra Pooyandeh & Insoo Sohn, 2023. "Smart Lithium-Ion Battery Monitoring in Electric Vehicles: An AI-Empowered Digital Twin Approach," Mathematics, MDPI, vol. 11(23), pages 1-37, December.
    2. Dorotea Dimitrova Angelova & Diego Carmona Fernández & Manuel Calderón Godoy & Juan Antonio Álvarez Moreno & Juan Félix González González, 2024. "A Review on Digital Twins and Its Application in the Modeling of Photovoltaic Installations," Energies, MDPI, vol. 17(5), pages 1-29, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mohamed Derbeli & Cristian Napole & Oscar Barambones, 2023. "A Fuzzy Logic Control for Maximum Power Point Tracking Algorithm Validated in a Commercial PV System," Energies, MDPI, vol. 16(2), pages 1-14, January.
    2. Kostas Bavarinos & Anastasios Dounis & Panagiotis Kofinas, 2021. "Maximum Power Point Tracking Based on Reinforcement Learning Using Evolutionary Optimization Algorithms," Energies, MDPI, vol. 14(2), pages 1-23, January.
    3. Jose Miguel Espi & Jaime Castello, 2019. "A Novel Fast MPPT Strategy for High Efficiency PV Battery Chargers," Energies, MDPI, vol. 12(6), pages 1-16, March.
    4. Jose Miguel Espi & Jaime Castello, 2019. "New Fast MPPT Method Based on a Power Slope Detector for Single Phase PV Inverters," Energies, MDPI, vol. 12(22), pages 1-20, November.
    5. Marco Balato & Carlo Petrarca, 2020. "The Impact of Reconfiguration on the Energy Performance of the Distributed Maximum Power Point Tracking Approach in PV Plants," Energies, MDPI, vol. 13(6), pages 1-19, March.
    6. Mohamed Derbeli & Cristian Napole & Oscar Barambones & Jesus Sanchez & Isidro Calvo & Pablo Fernández-Bustamante, 2021. "Maximum Power Point Tracking Techniques for Photovoltaic Panel: A Review and Experimental Applications," Energies, MDPI, vol. 14(22), pages 1-31, November.
    7. Camilo, Jones C. & Guedes, Tatiana & Fernandes, Darlan A. & Melo, J.D. & Costa, F.F. & Sguarezi Filho, Alfeu J., 2019. "A maximum power point tracking for photovoltaic systems based on Monod equation," Renewable Energy, Elsevier, vol. 130(C), pages 428-438.
    8. Emad M. Ahmed & Mokhtar Aly & Ahmed Elmelegi & Abdullah G. Alharbi & Ziad M. Ali, 2019. "Multifunctional Distributed MPPT Controller for 3P4W Grid-Connected PV Systems in Distribution Network with Unbalanced Loads," Energies, MDPI, vol. 12(24), pages 1-19, December.
    9. Tavakol Aghaei, Vahid & Ağababaoğlu, Arda & Bawo, Biram & Naseradinmousavi, Peiman & Yıldırım, Sinan & Yeşilyurt, Serhat & Onat, Ahmet, 2023. "Energy optimization of wind turbines via a neural control policy based on reinforcement learning Markov chain Monte Carlo algorithm," Applied Energy, Elsevier, vol. 341(C).
    10. Duberney Murillo-Yarce & José Alarcón-Alarcón & Marco Rivera & Carlos Restrepo & Javier Muñoz & Carlos Baier & Patrick Wheeler, 2020. "A Review of Control Techniques in Photovoltaic Systems," Sustainability, MDPI, vol. 12(24), pages 1-21, December.
    11. Dorotea Dimitrova Angelova & Diego Carmona Fernández & Manuel Calderón Godoy & Juan Antonio Álvarez Moreno & Juan Félix González González, 2024. "A Review on Digital Twins and Its Application in the Modeling of Photovoltaic Installations," Energies, MDPI, vol. 17(5), pages 1-29, March.
    12. Moacyr A. G. de Brito & Victor A. Prado & Edson A. Batista & Marcos G. Alves & Carlos A. Canesin, 2021. "Design Procedure to Convert a Maximum Power Point Tracking Algorithm into a Loop Control System," Energies, MDPI, vol. 14(15), pages 1-17, July.
    13. Eyal Amer & Alon Kuperman & Teuvo Suntio, 2019. "Direct Fixed-Step Maximum Power Point Tracking Algorithms with Adaptive Perturbation Frequency," Energies, MDPI, vol. 12(3), pages 1-16, January.
    14. Naamane Debdouche & Brahim Deffaf & Habib Benbouhenni & Zarour Laid & Mohamed I. Mosaad, 2023. "Direct Power Control for Three-Level Multifunctional Voltage Source Inverter of PV Systems Using a Simplified Super-Twisting Algorithm," Energies, MDPI, vol. 16(10), pages 1-32, May.
    15. Syed Zulqadar Hassan & Hui Li & Tariq Kamal & Uğur Arifoğlu & Sidra Mumtaz & Laiq Khan, 2017. "Neuro-Fuzzy Wavelet Based Adaptive MPPT Algorithm for Photovoltaic Systems," Energies, MDPI, vol. 10(3), pages 1-16, March.
    16. Noureddine Bouarroudj & Yehya Houam & Abdelhamid Djari & Vicente Feliu-Batlle & Abdelkader Lakhdari & Boualam Benlahbib, 2023. "A Linear Quadratic Integral Controller for PV-Module Voltage Regulation for the Purpose of Enhancing the Classical Incremental Conductance Algorithm," Energies, MDPI, vol. 16(11), pages 1-17, June.
    17. Alfredo Gil-Velasco & Carlos Aguilar-Castillo, 2021. "A Modification of the Perturb and Observe Method to Improve the Energy Harvesting of PV Systems under Partial Shading Conditions," Energies, MDPI, vol. 14(9), pages 1-12, April.
    18. Naoufel Zitouni & Rabiaa Gammoudi & Rim Attafi & Dhafer Mezgahni, 2023. "Developed and Intelligent Structure of a Control for PV Water Treatment System," Energies, MDPI, vol. 16(18), pages 1-30, September.
    19. Mohamed Louzazni & Daniel Tudor Cotfas & Petru Adrian Cotfas, 2020. "Management and Performance Control Analysis of Hybrid Photovoltaic Energy Storage System under Variable Solar Irradiation," Energies, MDPI, vol. 13(12), pages 1-23, June.
    20. Ernesto Bárcenas-Bárcenas & Diego R. Espinoza-Trejo & José A. Pecina-Sánchez & Héctor A. Álvarez-Macías & Isaac Compeán-Martínez & Ángel A. Vértiz-Hernández, 2023. "An improved Fractional MPPT Method by Using a Small Circle Approximation of the P–V Characteristic Curve," Mathematics, MDPI, vol. 11(3), pages 1-15, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:11:y:2023:i:9:p:2166-:d:1139635. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.