IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v15y2022i14p5117-d862092.html
   My bibliography  Save this article

Multi-Chamber Actuator Mode Selection through Reinforcement Learning–Simulations and Experiments

Author

Listed:
  • Henrique Raduenz

    (Division of Fluid and Mechatronics Systems, Linköping University, 581 83 Linköping, Sweden)

  • Liselott Ericson

    (Division of Fluid and Mechatronics Systems, Linköping University, 581 83 Linköping, Sweden)

  • Victor J. De Negri

    (Laboratory of Hydraulic and Pneumatic Systems, Federal University of Santa Catarina, Florianópolis 88040-900, Brazil)

  • Petter Krus

    (Division of Fluid and Mechatronics Systems, Linköping University, 581 83 Linköping, Sweden)

Abstract

This paper presents the development and implementation of a reinforcement learning agent as the mode selector for a multi-chamber actuator in a load-sensing architecture. The agent selects the mode of the actuator to minimise system energy losses. The agent was trained in a simulated environment and afterwards deployed to the real system. Simulation results indicated the capability of the agent to reduce energy consumption, while maintaining the actuation performance. Experimental results showed the capability of the agent to learn via simulation and to control the real system.

Suggested Citation

  • Henrique Raduenz & Liselott Ericson & Victor J. De Negri & Petter Krus, 2022. "Multi-Chamber Actuator Mode Selection through Reinforcement Learning–Simulations and Experiments," Energies, MDPI, vol. 15(14), pages 1-16, July.
    • Handle: RePEc:gam:jeners:v:15:y:2022:i:14:p:5117-:d:862092
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/15/14/5117/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/15/14/5117/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Zhang, Wei & Wang, Jixin & Liu, Yong & Gao, Guangzong & Liang, Siwen & Ma, Hongfeng, 2020. "Reinforcement learning-based intelligent energy management architecture for hybrid construction machinery," Applied Energy, Elsevier, vol. 275(C).
    2. Milos Vukovic & Roland Leifeld & Hubertus Murrenhoff, 2017. "Reducing Fuel Consumption in Hydraulic Excavators—A Comprehensive Analysis," Energies, MDPI, vol. 10(5), pages 1-25, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ryo Arai & Satoru Sakai & Akihiro Tatsuoka & Qin Zhang, 2021. "Analytical, Experimental, and Numerical Investigation of Energy in Hydraulic Cylinder Dynamics of Agriculture Scale Excavators," Energies, MDPI, vol. 14(19), pages 1-20, September.
    2. Andrea Vacca, 2018. "Energy Efficiency and Controllability of Fluid Power Systems," Energies, MDPI, vol. 11(5), pages 1-6, May.
    3. Daniel Egan & Qilun Zhu & Robert Prucka, 2023. "A Review of Reinforcement Learning-Based Powertrain Controllers: Effects of Agent Selection for Mixed-Continuity Control and Reward Formulation," Energies, MDPI, vol. 16(8), pages 1-31, April.
    4. Chongbo Jing & Junjie Zhou & Shihua Yuan & Siyuan Zhao, 2018. "Research on the Pressure Ratio Characteristics of a Swash Plate-Rotating Hydraulic Transformer," Energies, MDPI, vol. 11(6), pages 1-11, June.
    5. Jichao Liu & Yanyan Liang & Zheng Chen & Wenpeng Chen, 2023. "Energy Management Strategies for Hybrid Loaders: Classification, Comparison and Prospect," Energies, MDPI, vol. 16(7), pages 1-23, March.
    6. Paolo Casoli & Fabio Scolari & Carlo Maria Vescovini & Massimo Rundo, 2022. "Energy Comparison between a Load Sensing System and Electro-Hydraulic Solutions Applied to a 9-Ton Excavator," Energies, MDPI, vol. 15(7), pages 1-15, April.
    7. Chen, Chunyu & Cui, Mingjian & Fang, Xin & Ren, Bixing & Chen, Yang, 2020. "Load altering attack-tolerant defense strategy for load frequency control system," Applied Energy, Elsevier, vol. 280(C).
    8. Zhao, Liyuan & Yang, Ting & Li, Wei & Zomaya, Albert Y., 2022. "Deep reinforcement learning-based joint load scheduling for household multi-energy system," Applied Energy, Elsevier, vol. 324(C).
    9. Luis Javier Berne & Gustavo Raush & Pedro Javier Gamez-Montero & Pedro Roquet & Esteban Codina, 2021. "Multi-Point-of-View Energy Loss Analysis in a Refuse Truck Hydraulic System," Energies, MDPI, vol. 14(9), pages 1-24, May.
    10. Adam Wróblewski & Pavlo Krot & Radosław Zimroz & Timo Mayer & Jyri Peltola, 2023. "Review of Linear Electric Motor Hammers—An Energy-Saving and Eco-Friendly Solution in Industry," Energies, MDPI, vol. 16(2), pages 1-28, January.
    11. Luis Javier Berne & Gustavo Raush & Pedro Roquet & Pedro-Javier Gamez-Montero & Esteban Codina, 2022. "Graphic Method to Evaluate Power Requirements of a Hydraulic System Using Load-Holding Valves," Energies, MDPI, vol. 15(13), pages 1-23, June.
    12. Xiaofan Guo & Jacob Lengacher & Andrea Vacca, 2022. "A Variable Pressure Multi-Pressure Rail System Design for Agricultural Applications," Energies, MDPI, vol. 15(17), pages 1-25, August.
    13. Mirosław Przybysz & Marian Janusz Łopatka & Arkadiusz Rubiec & Piotr Krogul & Karol Cieślik & Marcin Małek, 2022. "Influence of Hydraulic Drivetrain Configuration on Kinematic Discrepancy and Energy Consumption during Obstacle Overcoming in a 6 × 6 All-Wheel Hydraulic Drive Vehicle," Energies, MDPI, vol. 15(17), pages 1-21, September.
    14. Daniele Beltrami & Paolo Iora & Laura Tribioli & Stefano Uberti, 2021. "Electrification of Compact Off-Highway Vehicles—Overview of the Current State of the Art and Trends," Energies, MDPI, vol. 14(17), pages 1-30, September.
    15. Václav Mergl & Zdravko Pandur & Jan Klepárník & Hrvoje Kopseak & Marin Bačić & Marijan Šušnjar, 2021. "Technical Solutions of Forest Machine Hybridization," Energies, MDPI, vol. 14(10), pages 1-14, May.
    16. Zhang, Wei & Wang, Jixin & Xu, Zhenyu & Shen, Yuying & Gao, Guangzong, 2022. "A generalized energy management framework for hybrid construction vehicles via model-based reinforcement learning," Energy, Elsevier, vol. 260(C).
    17. Kwangman An & Hyehyun Kang & Youngkuk An & Jinil Park & Jonghwa Lee, 2020. "Methodology of Excavator System Energy Flow-Down," Energies, MDPI, vol. 13(4), pages 1-19, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:15:y:2022:i:14:p:5117-:d:862092. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.