IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v311y2023i2p533-544.html
   My bibliography  Save this article

Partially observable Markov decision process-based optimal maintenance planning with time-dependent observations

Author

Listed:
  • Deep, Akash
  • Zhou, Shiyu
  • Veeramani, Dharmaraj
  • Chen, Yong

Abstract

The growing technological capability for real-time condition monitoring (CM) of industrial equipment has spurred significant interest in methods for optimal maintenance planning using CM signals. Existing approaches for maintenance policy development consider degradation to be either fully or partially observable. For the more general case of partial observability, it is usually assumed that the relationship between the underlying degradation process and the observed condition is time-invariant. In this paper, we address this major shortcoming by modeling observed CM signals through an underlying failure process wherein the linkage is time-dependent piecewise linear with jumps, and then utilizing a Partially Observed Markov Decision Process (POMDP) to determine the optimal maintenance strategy. We investigate the structure of the policy and show that, under certain conditions, a control-limit policy exists, i.e., a belief threshold exists beyond which the optimal action is to preventively maintain the unit. Finally, we present a case study based on battery resistance data and demonstrate that our modeling procedure offers a maintenance policy that is superior to those from other competing models.

Suggested Citation

  • Deep, Akash & Zhou, Shiyu & Veeramani, Dharmaraj & Chen, Yong, 2023. "Partially observable Markov decision process-based optimal maintenance planning with time-dependent observations," European Journal of Operational Research, Elsevier, vol. 311(2), pages 533-544.
  • Handle: RePEc:eee:ejores:v:311:y:2023:i:2:p:533-544
    DOI: 10.1016/j.ejor.2023.05.022
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221723003867
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ejor.2023.05.022?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Akash Deep & Shiyu Zhou & Dharmaraj Veeramani, 2022. "A data-driven recurrent event model for system degradation with imperfect maintenance actions," IISE Transactions, Taylor & Francis Journals, vol. 54(3), pages 271-285, March.
    2. Chen, Zhen & Li, Yaping & Xia, Tangbin & Pan, Ershun, 2019. "Hidden Markov model with auto-correlated observations for remaining useful life prediction and optimal maintenance policy," Reliability Engineering and System Safety, Elsevier, vol. 184(C), pages 123-136.
    3. Bautista, Lucía & Castro, Inma T. & Landesa, Luis, 2022. "Condition-based maintenance for a system subject to multiple degradation processes with stochastic arrival intensity," European Journal of Operational Research, Elsevier, vol. 302(2), pages 560-574.
    4. Donald Rosenfield, 1976. "Markovian Deterioration with Uncertain Information," Operations Research, INFORMS, vol. 24(1), pages 141-155, February.
    5. Alaswad, Suzan & Xiang, Yisha, 2017. "A review on condition-based maintenance optimization models for stochastically deteriorating system," Reliability Engineering and System Safety, Elsevier, vol. 157(C), pages 54-63.
    6. Salman Jahani & Raed Kontar & Shiyu Zhou & Dharmaraj Veeramani, 2020. "Remaining useful life prediction based on degradation signals using monotonic B-splines with infinite support," IISE Transactions, Taylor & Francis Journals, vol. 52(5), pages 537-554, May.
    7. Chen, Nan & Ye, Zhi-Sheng & Xiang, Yisha & Zhang, Linmiao, 2015. "Condition-based maintenance using the inverse Gaussian degradation model," European Journal of Operational Research, Elsevier, vol. 243(1), pages 190-199.
    8. Nan Chen & Kwok Tsui, 2013. "Condition monitoring and remaining useful life prediction using degradation signals: revisited," IISE Transactions, Taylor & Francis Journals, vol. 45(9), pages 939-952.
    9. John A. Flory & Jeffrey P. Kharoufeh & David T. Abdul‐Malak, 2015. "Optimal replacement of continuously degrading systems in partially observed environments," Naval Research Logistics (NRL), John Wiley & Sons, vol. 62(5), pages 395-415, August.
    10. Wang, Hongzhou, 2002. "A survey of maintenance policies of deteriorating systems," European Journal of Operational Research, Elsevier, vol. 139(3), pages 469-489, June.
    11. Schouten, Thijs Nicolaas & Dekker, Rommert & Hekimoğlu, Mustafa & Eruguz, Ayse Sena, 2022. "Maintenance optimization for a single wind turbine component under time-varying costs," European Journal of Operational Research, Elsevier, vol. 300(3), pages 979-991.
    12. Qiang Zhou & Junbo Son & Shiyu Zhou & Xiaofeng Mao & Mutasim Salman, 2014. "Remaining useful life prediction of individual units subject to hard failure," IISE Transactions, Taylor & Francis Journals, vol. 46(10), pages 1017-1030, October.
    13. Lu, Biao & Chen, Zhen & Zhao, Xufeng, 2021. "Data-driven dynamic predictive maintenance for a manufacturing system with quality deterioration and online sensors," Reliability Engineering and System Safety, Elsevier, vol. 212(C).
    14. Rocchetta, R. & Bellani, L. & Compare, M. & Zio, E. & Patelli, E., 2019. "A reinforcement learning framework for optimal operation and maintenance of power grids," Applied Energy, Elsevier, vol. 241(C), pages 291-301.
    15. Donald Rosenfield, 1976. "Markovian Deterioration With Uncertain Information — A More General Model," Naval Research Logistics Quarterly, John Wiley & Sons, vol. 23(3), pages 389-405, September.
    16. Ohnishi, Masamitsu & Kawai, Hajime & Mine, Hisashi, 1986. "An optimal inspection and replacement policy under incomplete state information," European Journal of Operational Research, Elsevier, vol. 27(1), pages 117-128, October.
    17. Francesco Cartella & Jan Lemeire & Luca Dimiccoli & Hichem Sahli, 2015. "Hidden Semi-Markov Models for Predictive Maintenance," Mathematical Problems in Engineering, Hindawi, vol. 2015, pages 1-23, February.
    18. Nguyen, Khanh T. P. & Do, Phuc & Huynh, Khac Tuan & Bérenguer, Christophe & Grall, Antoine, 2019. "Joint optimization of monitoring quality and replacement decisions in condition-based maintenance," Reliability Engineering and System Safety, Elsevier, vol. 189(C), pages 177-195.
    19. Lisa M. Maillart & Ludmila Zheltova, 2007. "Structured maintenance policies on interior sample paths," Naval Research Logistics (NRL), John Wiley & Sons, vol. 54(6), pages 645-655, September.
    20. William S. Lovejoy, 1987. "Some Monotonicity Results for Partially Observed Markov Decision Processes," Operations Research, INFORMS, vol. 35(5), pages 736-743, October.
    21. KarabaÄŸ, Oktay & Eruguz, Ayse Sena & Basten, Rob, 2020. "Integrated optimization of maintenance interventions and spare part selection for a partially observable multi-component system," Reliability Engineering and System Safety, Elsevier, vol. 200(C).
    22. Cha, Ji Hwan & Finkelstein, Maxim & Levitin, Gregory, 2018. "Bivariate preventive maintenance of systems with lifetimes dependent on a random shock process," European Journal of Operational Research, Elsevier, vol. 266(1), pages 122-134.
    23. Edward J. Sondik, 1978. "The Optimal Control of Partially Observable Markov Processes over the Infinite Horizon: Discounted Costs," Operations Research, INFORMS, vol. 26(2), pages 282-304, April.
    24. Moghaddass, Ramin & Zuo, Ming J., 2014. "An integrated framework for online diagnostic and prognostic health monitoring using a multistate deterioration process," Reliability Engineering and System Safety, Elsevier, vol. 124(C), pages 92-104.
    25. Cyrus Derman, 1963. "Optimal Replacement and Maintenance Under Markovian Deterioration with Probability Bounds on Failure," Management Science, INFORMS, vol. 9(3), pages 478-481, April.
    26. Olde Keizer, Minou C.A. & Flapper, Simme Douwe P. & Teunter, Ruud H., 2017. "Condition-based maintenance policies for systems with multiple dependent components: A review," European Journal of Operational Research, Elsevier, vol. 261(2), pages 405-420.
    27. Alaa H. Elwany & Nagi Z. Gebraeel & Lisa M. Maillart, 2011. "Structured Replacement Policies for Components with Complex Degradation Processes and Dedicated Sensors," Operations Research, INFORMS, vol. 59(3), pages 684-695, June.
    28. Mosayebi Omshi, E. & Grall, A. & Shemehsavar, S., 2020. "A dynamic auto-adaptive predictive maintenance policy for degradation with unknown parameters," European Journal of Operational Research, Elsevier, vol. 282(1), pages 81-92.
    29. Zhen Chen & Tangbin Xia & Ershun Pan, 2017. "Optimal multi-level classification and preventive maintenance policy for highly reliable products," International Journal of Production Research, Taylor & Francis Journals, vol. 55(8), pages 2232-2250, April.
    30. Chiel van Oosterom & Hao Peng & Geert-Jan van Houtum, 2017. "Maintenance optimization for a Markovian deteriorating system with population heterogeneity," IISE Transactions, Taylor & Francis Journals, vol. 49(1), pages 96-109, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Xu, Jianyu & Liu, Bin & Zhao, Xiujie & Wang, Xiao-Lin, 2024. "Online reinforcement learning for condition-based group maintenance using factored Markov decision processes," European Journal of Operational Research, Elsevier, vol. 315(1), pages 176-190.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hao Zhang & Weihua Zhang, 2023. "Analytical Solution to a Partially Observable Machine Maintenance Problem with Obvious Failures," Management Science, INFORMS, vol. 69(7), pages 3993-4015, July.
    2. de Jonge, Bram & Scarf, Philip A., 2020. "A review on maintenance optimization," European Journal of Operational Research, Elsevier, vol. 285(3), pages 805-824.
    3. Cai, Yue & Teunter, Ruud H. & de Jonge, Bram, 2023. "A data-driven approach for condition-based maintenance optimization," European Journal of Operational Research, Elsevier, vol. 311(2), pages 730-738.
    4. KarabaÄŸ, Oktay & Eruguz, Ayse Sena & Basten, Rob, 2020. "Integrated optimization of maintenance interventions and spare part selection for a partially observable multi-component system," Reliability Engineering and System Safety, Elsevier, vol. 200(C).
    5. Chiel van Oosterom & Lisa M. Maillart & Jeffrey P. Kharoufeh, 2017. "Optimal maintenance policies for a safety‐critical system and its deteriorating sensor," Naval Research Logistics (NRL), John Wiley & Sons, vol. 64(5), pages 399-417, August.
    6. Zhang, Zhengxin & Si, Xiaosheng & Hu, Changhua & Lei, Yaguo, 2018. "Degradation data analysis and remaining useful life estimation: A review on Wiener-process-based methods," European Journal of Operational Research, Elsevier, vol. 271(3), pages 775-796.
    7. Andersen, Jesper Fink & Andersen, Anders Reenberg & Kulahci, Murat & Nielsen, Bo Friis, 2022. "A numerical study of Markov decision process algorithms for multi-component replacement problems," European Journal of Operational Research, Elsevier, vol. 299(3), pages 898-909.
    8. Shi, Yue & Zhu, Weihang & Xiang, Yisha & Feng, Qianmei, 2020. "Condition-based maintenance optimization for multi-component systems subject to a system reliability requirement," Reliability Engineering and System Safety, Elsevier, vol. 202(C).
    9. Giorgio, Massimiliano & Pulcini, Gianpaolo, 2024. "The effect of model misspecification of the bounded transformed gamma process on maintenance optimization," Reliability Engineering and System Safety, Elsevier, vol. 241(C).
    10. Dias, Luis & Leitão, Armando & Guimarães, Luis, 2021. "Resource definition and allocation for a multi-asset portfolio with heterogeneous degradation," Reliability Engineering and System Safety, Elsevier, vol. 213(C).
    11. Zheng, Meimei & Lin, Jie & Xia, Tangbin & Liu, Yu & Pan, Ershun, 2023. "Joint condition-based maintenance and spare provisioning policy for a K-out-of-N system with failures during inspection intervals," European Journal of Operational Research, Elsevier, vol. 308(3), pages 1220-1232.
    12. Junbo Son & Yeongin Kim & Shiyu Zhou, 2022. "Alerting patients via health information system considering trust-dependent patient adherence," Information Technology and Management, Springer, vol. 23(4), pages 245-269, December.
    13. Lisa M. Maillart & Ludmila Zheltova, 2007. "Structured maintenance policies on interior sample paths," Naval Research Logistics (NRL), John Wiley & Sons, vol. 54(6), pages 645-655, September.
    14. Liu, Bin & Pandey, Mahesh D. & Wang, Xiaolin & Zhao, Xiujie, 2021. "A finite-horizon condition-based maintenance policy for a two-unit system with dependent degradation processes," European Journal of Operational Research, Elsevier, vol. 295(2), pages 705-717.
    15. Mosayebi Omshi, E. & Grall, A. & Shemehsavar, S., 2020. "A dynamic auto-adaptive predictive maintenance policy for degradation with unknown parameters," European Journal of Operational Research, Elsevier, vol. 282(1), pages 81-92.
    16. David T. Abdul‐Malak & Jeffrey P. Kharoufeh & Lisa M. Maillart, 2019. "Maintaining systems with heterogeneous spare parts," Naval Research Logistics (NRL), John Wiley & Sons, vol. 66(6), pages 485-501, September.
    17. Huynh, K.T., 2021. "An adaptive predictive maintenance model for repairable deteriorating systems using inverse Gaussian degradation process," Reliability Engineering and System Safety, Elsevier, vol. 213(C).
    18. Zhao, Xiujie & Liu, Bin & Xu, Jianyu & Wang, Xiao-Lin, 2023. "Imperfect maintenance policies for warranted products under stochastic performance degradation," European Journal of Operational Research, Elsevier, vol. 308(1), pages 150-165.
    19. Liu, Xingchen & Sun, Qiuzhuang & Ye, Zhi-Sheng & Yildirim, Murat, 2021. "Optimal multi-type inspection policy for systems with imperfect online monitoring," Reliability Engineering and System Safety, Elsevier, vol. 207(C).
    20. Lee, Juseong & Mitici, Mihaela, 2022. "Multi-objective design of aircraft maintenance using Gaussian process learning and adaptive sampling," Reliability Engineering and System Safety, Elsevier, vol. 218(PA).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:311:y:2023:i:2:p:533-544. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.