IDEAS home Printed from https://ideas.repec.org/a/eee/reensy/v244y2024ics0951832023008359.html
   My bibliography  Save this article

Maintenance optimization of multi-unit balanced systems using deep reinforcement learning

Author

Listed:
  • Liu, Hengchang
  • Li, Bo
  • Yao, Fengming
  • Hu, Gexi
  • Xie, Lei

Abstract

Balanced systems are extensively utilized in various engineering fields. However, the study of maintenance optimization for balanced systems under dependent competing risks is very limited, and the existing method for orthodox multi-unit systems cannot solve the maintenance problem of balanced systems. Thus, we present a novel maintenance optimization method for systems with multiple balanced units. The units in the system are assumed to experience multiple failure processes due to continuous degradation and external shock damage. The system failure occurs in the following situations: (1) the deterioration levels of units on the symmetric positions exceed critical thresholds. (2) the deterioration level of any unit reaches the failure threshold. (3) the shock magnitude is greater than the hard failure threshold. By employing a customized deep reinforcement learning algorithm, we derive the optimal maintenance policy under a fixed planning horizon. Notably, our approach differs from previous research on maintenance optimization for balanced systems, the proposed method avoids the discretizing of the deterioration state. This significantly enhances computational efficiency, particularly when applied to multi-unit balanced systems. Numerous numerical studies illustrate the effectiveness of our proposed maintenance optimization method for multi-unit balanced systems.

Suggested Citation

  • Liu, Hengchang & Li, Bo & Yao, Fengming & Hu, Gexi & Xie, Lei, 2024. "Maintenance optimization of multi-unit balanced systems using deep reinforcement learning," Reliability Engineering and System Safety, Elsevier, vol. 244(C).
    • Handle: RePEc:eee:reensy:v:244:y:2024:i:c:s0951832023008359
    DOI: 10.1016/j.ress.2023.109921
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0951832023008359
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ress.2023.109921?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Najafi, Seyedvahid & Zheng, Rui & Lee, Chi-Guhn, 2021. "An optimal opportunistic maintenance policy for a two-unit series system with general repair using proportional hazards models," Reliability Engineering and System Safety, Elsevier, vol. 215(C).
    2. Yang, Ao & Qiu, Qingan & Zhu, Mingren & Cui, Lirong & Chen, Weilin & Chen, Jianhui, 2022. "Condition-based maintenance strategy for redundant systems with arbitrary structures using improved reinforcement learning," Reliability Engineering and System Safety, Elsevier, vol. 225(C).
    3. Fang, Chen & Cui, Lirong, 2020. "Reliability analysis for balanced engine systems with m sectors by considering start-up probability," Reliability Engineering and System Safety, Elsevier, vol. 197(C).
    4. Wang, Jingjing & Wang, Zongxi & Zheng, Rui, 2021. "Optimal inventory policy for a balanced system subject to hard failure," Reliability Engineering and System Safety, Elsevier, vol. 216(C).
    5. Wang, Siqi & Zhao, Xian & Wu, Congshan & Wang, Xiaoyue, 2023. "Joint optimization of multi-stage component reassignment and preventive maintenance for balanced systems considering imperfect maintenance," Reliability Engineering and System Safety, Elsevier, vol. 237(C).
    6. Mohammadi, Reza & He, Qing, 2022. "A deep reinforcement learning approach for rail renewal and maintenance planning," Reliability Engineering and System Safety, Elsevier, vol. 225(C).
    7. Wang, Xiaoyue & Ning, Ru & Zhao, Xian & Wu, Congshan, 2023. "Reliability assessments for two types of balanced systems with multi-state protective devices," Reliability Engineering and System Safety, Elsevier, vol. 229(C).
    8. Dingguo Hua & Elsayed A. Elsayed, 2018. "Reliability approximation of k-out-of-n pairs: G balanced systems with spatially distributed units," IISE Transactions, Taylor & Francis Journals, vol. 50(7), pages 616-626, July.
    9. Wang, Jingjing & Zheng, Rui & Lin, Tianran, 2022. "Maintenance modeling for balanced systems subject to two competing failure modes," Reliability Engineering and System Safety, Elsevier, vol. 225(C).
    10. Zhao, Xian & Wang, Siqi & Wang, Xiaoyue & Fan, Yu, 2020. "Multi-state balanced systems in a shock environment," Reliability Engineering and System Safety, Elsevier, vol. 193(C).
    11. Lirong Cui & Jianhui Chen & Xiangchen Li, 2019. "Balanced reliability systems under Markov processes," IISE Transactions, Taylor & Francis Journals, vol. 51(9), pages 1025-1035, September.
    12. Yangyao, Shi & Xinchen, Zhuang & Tianxiang, Yu & Zijian, Zhang, 2023. "Multi-state balance system reliability research considering load influence," Reliability Engineering and System Safety, Elsevier, vol. 233(C).
    13. Zhang, Nailong & Si, Wujun, 2020. "Deep reinforcement learning for condition-based maintenance planning of multi-component systems under dependent competing risks," Reliability Engineering and System Safety, Elsevier, vol. 203(C).
    14. Wang, Jingjing & Qiu, Qingan & Wang, Huanhuan & Lin, Cong, 2021. "Optimal condition-based preventive maintenance policy for balanced systems," Reliability Engineering and System Safety, Elsevier, vol. 211(C).
    15. Volodymyr Mnih & Koray Kavukcuoglu & David Silver & Andrei A. Rusu & Joel Veness & Marc G. Bellemare & Alex Graves & Martin Riedmiller & Andreas K. Fidjeland & Georg Ostrovski & Stig Petersen & Charle, 2015. "Human-level control through deep reinforcement learning," Nature, Nature, vol. 518(7540), pages 529-533, February.
    16. van Noortwijk, J.M., 2009. "A survey of the application of gamma processes in maintenance," Reliability Engineering and System Safety, Elsevier, vol. 94(1), pages 2-21.
    17. Wang, Xiaoyue & Zhao, Xian & Wang, Siqi & Sun, Leping, 2020. "Reliability and maintenance for performance-balanced systems operating in a shock environment," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    18. Wang, Jingjing & Miao, Yonghao, 2021. "Optimal preventive maintenance policy of the balanced system under the semi-Markov model," Reliability Engineering and System Safety, Elsevier, vol. 213(C).
    19. Olde Keizer, Minou C.A. & Flapper, Simme Douwe P. & Teunter, Ruud H., 2017. "Condition-based maintenance policies for systems with multiple dependent components: A review," European Journal of Operational Research, Elsevier, vol. 261(2), pages 405-420.
    20. Zheng, Meimei & Ye, Hongqing & Wang, Dong & Pan, Ershun, 2021. "Joint Optimization of Condition-Based Maintenance and Spare Parts Orders for Multi-Unit Systems with Dual Sourcing," Reliability Engineering and System Safety, Elsevier, vol. 210(C).
    21. Hu, Jiawen & Shen, Jingyuan & Shen, Lijuan, 2020. "Opportunistic maintenance for two-component series systems subject to dependent degradation and shock," Reliability Engineering and System Safety, Elsevier, vol. 201(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wu, Congshan & Zhao, Xian & Wang, Siqi & Song, Yanbo, 2022. "Reliability analysis of consecutive-k-out-of-r-from-n subsystems: F balanced systems with load sharing," Reliability Engineering and System Safety, Elsevier, vol. 228(C).
    2. Wang, Xiaoyue & Ning, Ru & Zhao, Xian & Wu, Congshan, 2023. "Reliability assessments for two types of balanced systems with multi-state protective devices," Reliability Engineering and System Safety, Elsevier, vol. 229(C).
    3. Gan, Shuyuan & Hu, Hengheng & Coit, David W., 2023. "Maintenance optimization considering the mutual dependence of the environment and system with decreasing effects of imperfect maintenance," Reliability Engineering and System Safety, Elsevier, vol. 235(C).
    4. Wang, Jingjing & Zheng, Rui & Lin, Tianran, 2022. "Maintenance modeling for balanced systems subject to two competing failure modes," Reliability Engineering and System Safety, Elsevier, vol. 225(C).
    5. Wang, Xiaoyue & Ning, Ru & Zhao, Xian & Zhou, Jian, 2022. "Reliability analyses of k-out-of-n: F capability-balanced systems in a multi-source shock environment," Reliability Engineering and System Safety, Elsevier, vol. 227(C).
    6. Zhou, Yifan & Li, Bangcheng & Lin, Tian Ran, 2022. "Maintenance optimisation of multicomponent systems using hierarchical coordinated reinforcement learning," Reliability Engineering and System Safety, Elsevier, vol. 217(C).
    7. Yangyao, Shi & Xinchen, Zhuang & Tianxiang, Yu & Zijian, Zhang, 2023. "Multi-state balance system reliability research considering load influence," Reliability Engineering and System Safety, Elsevier, vol. 233(C).
    8. Wang, Siqi & Zhao, Xian & Wu, Congshan & Wang, Xiaoyue, 2023. "Joint optimization of multi-stage component reassignment and preventive maintenance for balanced systems considering imperfect maintenance," Reliability Engineering and System Safety, Elsevier, vol. 237(C).
    9. Wang, Jingjing & Qiu, Qingan & Wang, Huanhuan & Lin, Cong, 2021. "Optimal condition-based preventive maintenance policy for balanced systems," Reliability Engineering and System Safety, Elsevier, vol. 211(C).
    10. Zhao, Xian & Wang, Chen & Wang, Siqi, 2024. "Reliability analysis of multi-state balanced systems with standby components switching mechanism," Reliability Engineering and System Safety, Elsevier, vol. 242(C).
    11. Fang, Chen & Chen, Jianhui & Qiu, Daizhen, 2024. "Reliability modeling for balanced systems considering mission abort policies," Reliability Engineering and System Safety, Elsevier, vol. 243(C).
    12. Fang, Chen & Cui, Lirong, 2021. "Reliability evaluation for balanced systems with auto-balancing mechanisms," Reliability Engineering and System Safety, Elsevier, vol. 213(C).
    13. Wang, Siqi & Zhao, Xian & Tian, Zhigang & Zuo, Ming J., 2021. "Optimum component reassignment for balanced systems with multi-state components operating in a shock environment," Reliability Engineering and System Safety, Elsevier, vol. 210(C).
    14. Lee, Juseong & Mitici, Mihaela, 2023. "Deep reinforcement learning for predictive aircraft maintenance using probabilistic Remaining-Useful-Life prognostics," Reliability Engineering and System Safety, Elsevier, vol. 230(C).
    15. Lee, Dongkyu & Song, Junho, 2023. "Risk-informed operation and maintenance of complex lifeline systems using parallelized multi-agent deep Q-network," Reliability Engineering and System Safety, Elsevier, vol. 239(C).
    16. Wu, Congshan & Zhao, Xian & Wang, Xiaoyue & Wang, Siqi, 2021. "Reliability analysis of performance-based balanced systems with common bus performance sharing," Reliability Engineering and System Safety, Elsevier, vol. 215(C).
    17. Najafi, Seyedvahid & Lee, Chi-Guhn, 2023. "A deep reinforcement learning approach for repair-based maintenance of multi-unit systems using proportional hazards model," Reliability Engineering and System Safety, Elsevier, vol. 234(C).
    18. Wang, Jingjing & Wang, Zongxi & Zheng, Rui, 2021. "Optimal inventory policy for a balanced system subject to hard failure," Reliability Engineering and System Safety, Elsevier, vol. 216(C).
    19. Finkelstein, Maxim & Cha, Ji Hwan & Langston, Amy, 2023. "Improving classical optimal age-replacement policies for degrading items," Reliability Engineering and System Safety, Elsevier, vol. 236(C).
    20. Ke Chen & Xian Zhao & Qingan Qiu, 2022. "Optimal Task Abort and Maintenance Policies Considering Time Redundancy," Mathematics, MDPI, vol. 10(9), pages 1-16, April.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:reensy:v:244:y:2024:i:c:s0951832023008359. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/reliability-engineering-and-system-safety .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.