IDEAS home Printed from https://ideas.repec.org/a/eee/reensy/v261y2025ics0951832025002960.html

Dynamic condition-based maintenance for shock systems based on damage evolutions using deep reinforcement learning

Author

Listed:
  • Sun, Yudao
  • Yin, Juan

Abstract

In the industry domain, maintenance tasks and resources need to be allocated to industrial systems to avoid unplanned downtime. We explore the dynamic condition-based maintenance strategy for systems comprising multiple components, in which each component undergoes external shocks along with time and is maintained individually. For each component, random shocks arrive following a homogeneous Poisson process, and the evolution of the component’s state is characterized using a Markov process. The dynamic condition-based maintenance policy for the developed shock system, depicted as a Markov decision process, is introduced. To minimize the overall system cost, the maintenance optimization problem is discussed to determine the most cost-effective maintenance actions. A tailored advantage actor-critic algorithm in deep reinforcement learning is proposed to address the challenge of high dimensionality. Finally, numerical examples demonstrate the efficiency of the proposed method in searching for optimal maintenance actions and reducing maintenance costs.

Suggested Citation

  • Sun, Yudao & Yin, Juan, 2025. "Dynamic condition-based maintenance for shock systems based on damage evolutions using deep reinforcement learning," Reliability Engineering and System Safety, Elsevier, vol. 261(C).
    • Handle: RePEc:eee:reensy:v:261:y:2025:i:c:s0951832025002960
    DOI: 10.1016/j.ress.2025.111095
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0951832025002960
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ress.2025.111095?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Masaaki Kijima & Toshio Nakagawa, 1991. "A cumulative damage shock model with imperfect preventive maintenance," Naval Research Logistics (NRL), John Wiley & Sons, vol. 38(2), pages 145-156, April.
    2. Mohammadi, Reza & He, Qing, 2022. "A deep reinforcement learning approach for rail renewal and maintenance planning," Reliability Engineering and System Safety, Elsevier, vol. 225(C).
    3. Liu, Hengchang & Li, Bo & Yao, Fengming & Hu, Gexi & Xie, Lei, 2024. "Maintenance optimization of multi-unit balanced systems using deep reinforcement learning," Reliability Engineering and System Safety, Elsevier, vol. 244(C).
    4. Qiao, Peirui & Ma, Yizhong & Luo, Ming & Shen, Jingyuan & Zhou, Hanting, 2024. "Reliability modeling and warranty optimization for products with self-healing under a dynamic shock environment," Reliability Engineering and System Safety, Elsevier, vol. 249(C).
    5. Chengjie Wang & Jian Liu & Qingyu Yang & Qingpei Hu & Dan Yu, 2023. "Recoverability effects on reliability assessment for accelerated degradation testing," IISE Transactions, Taylor & Francis Journals, vol. 55(7), pages 698-710, July.
    6. Liu, Bin & Pandey, Mahesh D. & Wang, Xiaolin & Zhao, Xiujie, 2021. "A finite-horizon condition-based maintenance policy for a two-unit system with dependent degradation processes," European Journal of Operational Research, Elsevier, vol. 295(2), pages 705-717.
    7. Xiao Wang & Hongwei Wang & Chao Qi, 2016. "Multi-agent reinforcement learning based maintenance policy for a resource constrained flow line system," Journal of Intelligent Manufacturing, Springer, vol. 27(2), pages 325-333, April.
    8. Yang, Li & Ye, Zhi-sheng & Lee, Chi-Guhn & Yang, Su-fen & Peng, Rui, 2019. "A two-phase preventive maintenance policy considering imperfect repair and postponed replacement," European Journal of Operational Research, Elsevier, vol. 274(3), pages 966-977.
    9. Li, Meiyan & Wu, Bei, 2024. "Optimal condition-based opportunistic maintenance policy for two-component systems considering common cause failure," Reliability Engineering and System Safety, Elsevier, vol. 250(C).
    10. Yang, Ao & Qiu, Qingan & Zhu, Mingren & Cui, Lirong & Chen, Weilin & Chen, Jianhui, 2022. "Condition-based maintenance strategy for redundant systems with arbitrary structures using improved reinforcement learning," Reliability Engineering and System Safety, Elsevier, vol. 225(C).
    11. Liu, Yu & Chen, Yiming & Jiang, Tao, 2020. "Dynamic selective maintenance optimization for multi-state systems over a finite horizon: A deep reinforcement learning approach," European Journal of Operational Research, Elsevier, vol. 283(1), pages 166-181.
    12. Ranjkesh, Somayeh Hamed & Hamadani, Ali Zeinal & Mahmoodi, Safieh, 2019. "A new cumulative shock model with damage and inter-arrival time dependency," Reliability Engineering and System Safety, Elsevier, vol. 192(C).
    13. Ferreira Neto, Waldomiro Alves & Virgínio Cavalcante, Cristiano Alexandre & Do, Phuc, 2024. "Deep reinforcement learning for maintenance optimization of a scrap-based steel production line," Reliability Engineering and System Safety, Elsevier, vol. 249(C).
    14. Lirong Cui & Fengming Kang, 2023. "Shocks models with damage effect evolutions following Markov processes," Journal of the Operational Research Society, Taylor & Francis Journals, vol. 74(1), pages 430-444, January.
    15. Cha, Ji Hwan & Finkelstein, Maxim, 2016. "New shock models based on the generalized Polya process," European Journal of Operational Research, Elsevier, vol. 251(1), pages 135-141.
    16. Volodymyr Mnih & Koray Kavukcuoglu & David Silver & Andrei A. Rusu & Joel Veness & Marc G. Bellemare & Alex Graves & Martin Riedmiller & Andreas K. Fidjeland & Georg Ostrovski & Stig Petersen & Charle, 2015. "Human-level control through deep reinforcement learning," Nature, Nature, vol. 518(7540), pages 529-533, February.
    17. Zhang, Nan & Cai, Kaiquan & Zhang, Jun & Wang, Tian, 2022. "A condition-based maintenance policy considering failure dependence and imperfect inspection for a two-component system," Reliability Engineering and System Safety, Elsevier, vol. 217(C).
    18. Yunyi Kang & Feng Ju, 2019. "Flexible preventative maintenance for serial production lines with multi-stage degrading machines and finite buffers," IISE Transactions, Taylor & Francis Journals, vol. 51(7), pages 777-791, July.
    19. Luo, Yi & Zhao, Xiujie & Liu, Bin & He, Shuguang, 2024. "Condition-based maintenance policy for systems under dynamic environment," Reliability Engineering and System Safety, Elsevier, vol. 246(C).
    20. Olde Keizer, Minou C.A. & Teunter, Ruud H. & Veldman, Jasper, 2016. "Clustering condition-based maintenance for systems with redundancy and economic dependencies," European Journal of Operational Research, Elsevier, vol. 251(2), pages 531-540.
    21. Chadjiconstantinidis, Stathis & Eryilmaz, Serkan, 2023. "Reliability of a mixed δ-shock model with a random change point in shock magnitude distribution and an optimal replacement policy," Reliability Engineering and System Safety, Elsevier, vol. 232(C).
    22. Phalguni Nanda & Prajamitra Bhuyan & Anup Dewanji, 2022. "Optimal replacement policy under cumulative damage model and strength degradation with applications," Annals of Operations Research, Springer, vol. 315(2), pages 1345-1371, August.
    23. Zhang, Zhengxin & Li, Huiqin & Li, Tianmei & Zhang, Jianxun & Si, Xiaosheng, 2024. "An optimal condition-based maintenance policy for nonlinear stochastic degrading systems," Reliability Engineering and System Safety, Elsevier, vol. 251(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Shi, Guannan & Zhang, Xiaohong & Zeng, Jianchao & Liao, Haitao & Gan, Jie & Wang, Jinhe & Wang, Zhijian, 2025. "A predictive maintenance framework based on real-time credibility evaluation of remaining useful life prediction results," Reliability Engineering and System Safety, Elsevier, vol. 264(PA).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhang, Wenyu & He, Shuguang & Zhang, Xiaohong & Zhao, Xing, 2024. "Joint optimization of job scheduling, condition-based maintenance planning, and spare parts ordering for degrading production systems," Reliability Engineering and System Safety, Elsevier, vol. 252(C).
    2. Yang, Ao & Qiu, Qingan & Zhu, Mingren & Cui, Lirong & Chen, Weilin & Chen, Jianhui, 2022. "Condition-based maintenance strategy for redundant systems with arbitrary structures using improved reinforcement learning," Reliability Engineering and System Safety, Elsevier, vol. 225(C).
    3. Zheng, Rui & Wu, Kai & Lu, Shaojun & Li, Mengmeng, 2025. "Optimal condition-based replacement policy with unknown degradation parameters," Reliability Engineering and System Safety, Elsevier, vol. 264(PA).
    4. Zhao, Sangqi & Wei, Yian & Li, Yang & Cheng, Yao, 2026. "A multi-agent reinforcement learning (MARL) framework for designing an optimal state-specific hybrid maintenance policy for a series k-out-of-n load-sharing system," Reliability Engineering and System Safety, Elsevier, vol. 265(PA).
    5. Chen, Jiangxi & Zhou, Xiaojun, 2026. "A GNN-boosted reinforcement learning framework for maintenance optimization in multi-dependency manufacturing system," Reliability Engineering and System Safety, Elsevier, vol. 269(C).
    6. Chen, Jiangxi & Zhou, Xiaojun, 2025. "Reinforcement learning based maintenance scheduling of flexible multi-machine manufacturing systems with varying interactive degradation," Reliability Engineering and System Safety, Elsevier, vol. 260(C).
    7. Wang, Changwen & Ning, Guolong & Deng, Qianwang & Liu, Rui & Luo, Qiang, 2025. "Joint optimization of cooperative maintenance and inventory control for multiple k-out-of-n: F systems considering component interchange and shared spare parts," Reliability Engineering and System Safety, Elsevier, vol. 262(C).
    8. Zhao, Sangqi & Wei, Yian & Cheng, Yao & Li, Yang, 2025. "A state-specific joint size, maintenance, and inventory policy for a k-out-of-n load-sharing system subject to self-announcing failures," Reliability Engineering and System Safety, Elsevier, vol. 257(PB).
    9. Zhou, Yifan & Li, Bangcheng & Lin, Tian Ran, 2022. "Maintenance optimisation of multicomponent systems using hierarchical coordinated reinforcement learning," Reliability Engineering and System Safety, Elsevier, vol. 217(C).
    10. Zhu, Yunxin & Zheng, Meimei & Su, Zhiyun & Xia, Tangbin & Lin, Jie & Pan, Ershun, 2025. "Deep reinforcement learning for joint optimization of maintenance and spare parts ordering considering spare parts supply uncertainty," Reliability Engineering and System Safety, Elsevier, vol. 264(PB).
    11. Kermanie, Elham Zakaryaie & Aghaie, Abdollah & Shahriari, Hamid, 2026. "Reliability estimation under autocorrelated cumulative shock damage: A time series modeling approach," Reliability Engineering and System Safety, Elsevier, vol. 267(PA).
    12. Ning, Ru & Wang, Xiaoyue & Zhao, Xian & Li, Ziyue, 2024. "Joint optimization of preventive maintenance and triggering mechanism for k-out-of-n: F systems with protective devices based on periodic inspection," Reliability Engineering and System Safety, Elsevier, vol. 251(C).
    13. Andersen, Jesper Fink & Nielsen, Bo Friis, 2025. "A comparative study of time-based maintenance and condition-based maintenance for multi-component systems," Reliability Engineering and System Safety, Elsevier, vol. 256(C).
    14. Liu, Hengchang & Li, Bo & Yao, Fengming & Hu, Gexi & Xie, Lei, 2024. "Maintenance optimization of multi-unit balanced systems using deep reinforcement learning," Reliability Engineering and System Safety, Elsevier, vol. 244(C).
    15. Lee, Dongkyu & Song, Junho, 2023. "Risk-informed operation and maintenance of complex lifeline systems using parallelized multi-agent deep Q-network," Reliability Engineering and System Safety, Elsevier, vol. 239(C).
    16. Wei, Yian & Li, Anchi & Li, Yang & Cheng, Yao, 2026. "An optimal predictive inspection and maintenance policy for a multi-state system: A belief-based SMDP approach," Reliability Engineering and System Safety, Elsevier, vol. 265(PB).
    17. Xian Zhao & Rong Li & Yu Fan & Qingan Qiu, 2022. "Reliability modeling for multi-state systems with a protective device considering multiple triggering mechanism," Journal of Risk and Reliability, , vol. 236(1), pages 173-193, February.
    18. Zhang, Fengxia & Liao, Haitao & Shen, Jingyuan & Ma, Yizhong, 2024. "Optimal maintenance over a finite time horizon for a system under imperfect inspection and dynamic working environment," Reliability Engineering and System Safety, Elsevier, vol. 246(C).
    19. Meng, Yinghao & Qi, Faqun & Liu, Bowen & Yin, Zhen & Ye, Shuwen & Zhang, Haosen, 2025. "Joint optimization of maintenance strategy and buffer capacity for a multi-state buffered serial system considering self-repair actions," Reliability Engineering and System Safety, Elsevier, vol. 262(C).
    20. Shang, Lijun & Liu, Baoliang & Peng, Rui, 2025. "Designing random collaborative warranty and customizing maintenance strategies for systems subject to mission cycles," Reliability Engineering and System Safety, Elsevier, vol. 257(PA).

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:reensy:v:261:y:2025:i:c:s0951832025002960. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/reliability-engineering-and-system-safety .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.