Operational readiness-oriented condition-based maintenance and spare parts optimization for multi-state systems

In many military scenarios, engineered systems are required to remain satisfied with operational readiness to respond to unexpected tasks. However, the degradation caused by daily usage inherently decreases the operational readiness of these systems. Condition-based maintenance is an efficient strategy that can recover the system operational readiness by restoring the system condition. On the other hand, the activities of maintenance are often constrained by spare parts ordering. Most existing research only pays attention on the daily work and ignores the requirement of operational readiness. In this paper, a novel reinforcement learning (RL) based condition-based maintenance and spare parts optimization method for multiple unit multi-state systems (MSS) is proposed, aimed at minimizing long-term cost rate considering the requirement of operational readiness and daily work. The resulting joint decision-making problem is formulated as a discrete-time discrete-state Markov decision process (MDP) and a customized architecture of value iteration algorithm embedded with a stratified sampling Monte Carlo (SSMC) method is introduced. A real case of armored vehicles in a military base is provided to prove the effectiveness of our method. From comparative experiments and sensitivity analysis of serval examples, several interesting suggestions are presented.