IDEAS home Printed from https://ideas.repec.org/a/eee/reensy/v255y2025ics0951832024007300.html
   My bibliography  Save this article

Condition-based maintenance for multi-state systems with prognostic and deep reinforcement learning

Author

Listed:
  • Zhang, Huixian
  • Wei, Xiukun
  • Liu, Zhiqiang
  • Ding, Yaning
  • Guan, Qingluan

Abstract

The utilization of prognostic information in practical engineering is increasing with the development of technology and predictive modeling. Current research on maintenance strategies for complex multi-state systems often neglects prognostic information or assumes complete availability of all component information. This paper investigates the joint maintenance strategies based on condition-based maintenance for complex multi-state systems, in which the predicted remaining useful life of some components is known. Firstly, a maintenance strategy framework is developed and the joint maintenance strategy is proposed for the studied problem. Then the deterioration process of the component, the imperfect maintenance, and prediction error models are constructed. The optimization problem is modeled as a Markov Decision Process to minimize the maintenance cost, and the system reliability constraints are established by using the universal generating function method. In addition, a deep Q-network is designed to solve the optimal maintenance policy. Finally, the traction system of a metro train is taken as an example to verify the applicability of the model and algorithm. The results show that the proposed maintenance strategy reduces the maintenance cost compared to the current maintenance strategy for both fixed maintenance intervals and dynamic maintenance intervals.

Suggested Citation

  • Zhang, Huixian & Wei, Xiukun & Liu, Zhiqiang & Ding, Yaning & Guan, Qingluan, 2025. "Condition-based maintenance for multi-state systems with prognostic and deep reinforcement learning," Reliability Engineering and System Safety, Elsevier, vol. 255(C).
    • Handle: RePEc:eee:reensy:v:255:y:2025:i:c:s0951832024007300
    DOI: 10.1016/j.ress.2024.110659
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0951832024007300
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ress.2024.110659?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Yaqiong Lv & Pan Zheng & Jiabei Yuan & Xiaohua Cao, 2023. "A Predictive Maintenance Strategy for Multi-Component Systems Based on Components’ Remaining Useful Life Prediction," Mathematics, MDPI, vol. 11(18), pages 1-23, September.
    2. Kijima, Masaaki & Morimura, Hidenori & Suzuki, Yasusuke, 1988. "Periodical replacement problem without assuming minimal repair," European Journal of Operational Research, Elsevier, vol. 37(2), pages 194-203, November.
    3. Yang, Ao & Qiu, Qingan & Zhu, Mingren & Cui, Lirong & Chen, Weilin & Chen, Jianhui, 2022. "Condition-based maintenance strategy for redundant systems with arbitrary structures using improved reinforcement learning," Reliability Engineering and System Safety, Elsevier, vol. 225(C).
    4. Tseremoglou, Iordanis & Santos, Bruno F., 2024. "Condition-Based Maintenance scheduling of an aircraft fleet under partial observability: A Deep Reinforcement Learning approach," Reliability Engineering and System Safety, Elsevier, vol. 241(C).
    5. He, Rui & Tian, Zhigang & Wang, Yifei & Zuo, Mingjian & Guo, Ziwei, 2023. "Condition-based maintenance optimization for multi-component systems considering prognostic information and degraded working efficiency," Reliability Engineering and System Safety, Elsevier, vol. 234(C).
    6. Arias Chao, Manuel & Kulkarni, Chetan & Goebel, Kai & Fink, Olga, 2022. "Fusing physics-based and deep learning models for prognostics," Reliability Engineering and System Safety, Elsevier, vol. 217(C).
    7. Lee, Juseong & Mitici, Mihaela, 2023. "Deep reinforcement learning for predictive aircraft maintenance using probabilistic Remaining-Useful-Life prognostics," Reliability Engineering and System Safety, Elsevier, vol. 230(C).
    8. Wakiru, James & Pintelon, Liliane & Muchiri, Peter N. & Chemweno, Peter K. & Mburu, Stanley, 2020. "Towards an innovative lubricant condition monitoring strategy for maintenance of ageing multi-unit systems," Reliability Engineering and System Safety, Elsevier, vol. 204(C).
    9. Nguyen, Khanh T.P. & Medjaher, Kamal, 2019. "A new dynamic predictive maintenance framework using deep learning for failure prognostics," Reliability Engineering and System Safety, Elsevier, vol. 188(C), pages 251-262.
    10. Pourkarim Guilani, Pedram & Sharifi, Mani & Niaki, S.T.A. & Zaretalab, Arash, 2014. "Reliability evaluation of non-reparable three-state systems using Markov model and its comparison with the UGF and the recursive methods," Reliability Engineering and System Safety, Elsevier, vol. 129(C), pages 29-35.
    11. Hashemi, M. & Asadi, M. & Zarezadeh, S., 2020. "Optimal maintenance policies for coherent systems with multi-type components," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    12. Ruiz-Castro, Juan Eloy, 2020. "A complex multi-state k-out-of-n: G system with preventive maintenance and loss of units," Reliability Engineering and System Safety, Elsevier, vol. 197(C).
    13. Zhao, Yunfei & Smidts, Carol, 2022. "Reinforcement learning for adaptive maintenance policy optimization under imperfect knowledge of the system degradation model and partial observability of system states," Reliability Engineering and System Safety, Elsevier, vol. 224(C).
    14. Shahraki, Ameneh Forouzandeh & Yadav, Om Prakash & Vogiatzis, Chrysafis, 2020. "Selective maintenance optimization for multi-state systems considering stochastically dependent components and stochastic imperfect maintenance actions," Reliability Engineering and System Safety, Elsevier, vol. 196(C).
    15. Khaled Akkad & David He, 2023. "A dynamic mode decomposition based deep learning technique for prognostics," Journal of Intelligent Manufacturing, Springer, vol. 34(5), pages 2207-2224, June.
    16. de Pater, Ingeborg & Reijns, Arthur & Mitici, Mihaela, 2022. "Alarm-based predictive maintenance scheduling for aircraft engines with imperfect Remaining Useful Life prognostics," Reliability Engineering and System Safety, Elsevier, vol. 221(C).
    17. Zhou, Kai-Li & Cheng, De-Jun & Zhang, Han-Bing & Hu, Zhong-tai & Zhang, Chun-Yan, 2023. "Deep learning-based intelligent multilevel predictive maintenance framework considering comprehensive cost," Reliability Engineering and System Safety, Elsevier, vol. 237(C).
    18. Najafi, Seyedvahid & Lee, Chi-Guhn, 2023. "A deep reinforcement learning approach for repair-based maintenance of multi-unit systems using proportional hazards model," Reliability Engineering and System Safety, Elsevier, vol. 234(C).
    19. Li, Heping & Deloux, Estelle & Dieulle, Laurence, 2016. "A condition-based maintenance policy for multi-component systems with Lévy copulas dependence," Reliability Engineering and System Safety, Elsevier, vol. 149(C), pages 44-55.
    20. Volodymyr Mnih & Koray Kavukcuoglu & David Silver & Andrei A. Rusu & Joel Veness & Marc G. Bellemare & Alex Graves & Martin Riedmiller & Andreas K. Fidjeland & Georg Ostrovski & Stig Petersen & Charle, 2015. "Human-level control through deep reinforcement learning," Nature, Nature, vol. 518(7540), pages 529-533, February.
    21. Zhuang, Liangliang & Xu, Ancha & Wang, Xiao-Lin, 2023. "A prognostic driven predictive maintenance framework based on Bayesian deep learning," Reliability Engineering and System Safety, Elsevier, vol. 234(C).
    22. Mitici, Mihaela & de Pater, Ingeborg & Barros, Anne & Zeng, Zhiguo, 2023. "Dynamic predictive maintenance for multiple components using data-driven probabilistic RUL prognostics: The case of turbofan engines," Reliability Engineering and System Safety, Elsevier, vol. 234(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kamariotis, Antonios & Tatsis, Konstantinos & Chatzi, Eleni & Goebel, Kai & Straub, Daniel, 2024. "A metric for assessing and optimizing data-driven prognostic algorithms for predictive maintenance," Reliability Engineering and System Safety, Elsevier, vol. 242(C).
    2. Dehghan Shoorkand, Hassan & Nourelfath, Mustapha & Hajji, Adnène, 2024. "A hybrid CNN-LSTM model for joint optimization of production and imperfect predictive maintenance planning," Reliability Engineering and System Safety, Elsevier, vol. 241(C).
    3. Lee, Juseong & Mitici, Mihaela, 2023. "Deep reinforcement learning for predictive aircraft maintenance using probabilistic Remaining-Useful-Life prognostics," Reliability Engineering and System Safety, Elsevier, vol. 230(C).
    4. Keshun, You & Guangqi, Qiu & Yingkui, Gu, 2024. "Optimizing prior distribution parameters for probabilistic prediction of remaining useful life using deep learning," Reliability Engineering and System Safety, Elsevier, vol. 242(C).
    5. Basora, Luis & Viens, Arthur & Chao, Manuel Arias & Olive, Xavier, 2025. "A benchmark on uncertainty quantification for deep learning prognostics," Reliability Engineering and System Safety, Elsevier, vol. 253(C).
    6. Li, Xiao Yan & Cheng, De Jun & Fang, Xi Feng & Zhang, Chun Yan & Wang, Yu Feng, 2024. "A novel data augmentation strategy for aeroengine multitask prognosis based on degradation behavior extrapolation and diversity-usability trade-off," Reliability Engineering and System Safety, Elsevier, vol. 249(C).
    7. Pliego Marugán, Alberto & Pinar-Pérez, Jesús M. & García Márquez, Fausto Pedro, 2024. "A reinforcement learning agent for maintenance of deteriorating systems with increasingly imperfect repairs," Reliability Engineering and System Safety, Elsevier, vol. 252(C).
    8. Ferreira Neto, Waldomiro Alves & Virgínio Cavalcante, Cristiano Alexandre & Do, Phuc, 2024. "Deep reinforcement learning for maintenance optimization of a scrap-based steel production line," Reliability Engineering and System Safety, Elsevier, vol. 249(C).
    9. Zhuang, Liangliang & Xu, Ancha & Wang, Xiao-Lin, 2023. "A prognostic driven predictive maintenance framework based on Bayesian deep learning," Reliability Engineering and System Safety, Elsevier, vol. 234(C).
    10. Yang, Li & Zhou, Shihan & Ma, Xiaobing & Chen, Yi & Jia, Heping & Dai, Wei, 2024. "Group machinery intelligent maintenance: Adaptive health prediction and global dynamic maintenance decision-making," Reliability Engineering and System Safety, Elsevier, vol. 252(C).
    11. Najafi, Seyedvahid & Lee, Chi-Guhn, 2023. "A deep reinforcement learning approach for repair-based maintenance of multi-unit systems using proportional hazards model," Reliability Engineering and System Safety, Elsevier, vol. 234(C).
    12. Pan, Junlin & Sun, Bo & Wu, Zeyu & Yi, Zechen & Feng, Qiang & Ren, Yi & Wang, Zili, 2024. "Probabilistic remaining useful life prediction without lifetime labels: A Bayesian deep learning and stochastic process fusion method," Reliability Engineering and System Safety, Elsevier, vol. 250(C).
    13. Zhang, Qin & Liu, Yu & Xiang, Yisha & Xiahou, Tangfan, 2024. "Reinforcement learning in reliability and maintenance optimization: A tutorial," Reliability Engineering and System Safety, Elsevier, vol. 251(C).
    14. Li, Yuanfu & Chen, Yao & Hu, Zhenchao & Zhang, Huisheng, 2023. "Remaining useful life prediction of aero-engine enabled by fusing knowledge and deep learning models," Reliability Engineering and System Safety, Elsevier, vol. 229(C).
    15. Zhang, Nan & Cai, Kaiquan & Zhang, Jun & Wang, Tian, 2022. "A condition-based maintenance policy considering failure dependence and imperfect inspection for a two-component system," Reliability Engineering and System Safety, Elsevier, vol. 217(C).
    16. Liu, Yu & Chen, Yiming & Jiang, Tao, 2020. "Dynamic selective maintenance optimization for multi-state systems over a finite horizon: A deep reinforcement learning approach," European Journal of Operational Research, Elsevier, vol. 283(1), pages 166-181.
    17. Tian, Yuan & Han, Minghao & Kulkarni, Chetan & Fink, Olga, 2022. "A prescriptive Dirichlet power allocation policy with deep reinforcement learning," Reliability Engineering and System Safety, Elsevier, vol. 224(C).
    18. Chadha, Mayank & Ramancha, Mukesh K. & Vega, Manuel A. & Conte, Joel P. & Todd, Michael D., 2023. "The modeling of risk perception in the use of structural health monitoring information for optimal maintenance decisions," Reliability Engineering and System Safety, Elsevier, vol. 229(C).
    19. Xiao, Dasheng & Lin, Zhifu & Yu, Aiyang & Tang, Ke & Xiao, Hong, 2024. "Data-driven method embedded physical knowledge for entire lifecycle degradation monitoring in aircraft engines," Reliability Engineering and System Safety, Elsevier, vol. 247(C).
    20. Zhengyang Fan & Wanru Li & Kuo-Chu Chang, 2023. "A Bidirectional Long Short-Term Memory Autoencoder Transformer for Remaining Useful Life Estimation," Mathematics, MDPI, vol. 11(24), pages 1-17, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:reensy:v:255:y:2025:i:c:s0951832024007300. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/reliability-engineering-and-system-safety .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.