IDEAS home Printed from https://ideas.repec.org/a/sae/risrel/v234y2020i1p52-62.html
   My bibliography  Save this article

A reinforcement learning approach to optimal part flow management for gas turbine maintenance

Author

Listed:
  • Michele Compare
  • Luca Bellani
  • Enrico Cobelli
  • Enrico Zio
  • Francesco Annunziata
  • Fausto Carlevaro
  • Marzia Sepe

Abstract

We consider the maintenance process of gas turbines used in the Oil and Gas industry: the capital parts are first removed from the gas turbines and replaced by parts of the same type taken from the warehouse; then, they are repaired at the workshop and returned to the warehouse for use in future maintenance events. Experience-based rules are used to manage the flow of the parts for a profitable gas turbine operation. In this article, we formalize the part flow management as a sequential decision problem and propose reinforcement learning for its solution. An application to a scaled-down case study derived from real industrial practice shows that reinforcement learning can find policies outperforming those based on experience-based rules.

Suggested Citation

  • Michele Compare & Luca Bellani & Enrico Cobelli & Enrico Zio & Francesco Annunziata & Fausto Carlevaro & Marzia Sepe, 2020. "A reinforcement learning approach to optimal part flow management for gas turbine maintenance," Journal of Risk and Reliability, , vol. 234(1), pages 52-62, February.
    • Handle: RePEc:sae:risrel:v:234:y:2020:i:1:p:52-62
    DOI: 10.1177/1748006X19869750
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/1748006X19869750
    Download Restriction: no
    File URL: https://libkey.io/10.1177/1748006X19869750?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Jin, Tongdan & Tian, Zhigang & Xie, Min, 2015. "A game-theoretical approach for optimizing maintenance, spares and service capacity in performance contracting," International Journal of Production Economics, Elsevier, vol. 161(C), pages 31-43.
    2. Wang, Wenbin, 2010. "A model for maintenance service contract design, negotiation and optimization," European Journal of Operational Research, Elsevier, vol. 201(1), pages 239-246, February.
    3. Van Horenbeek, Adriaan & Buré, Jasmine & Cattrysse, Dirk & Pintelon, Liliane & Vansteenwegen, Pieter, 2013. "Joint maintenance and inventory optimization systems: A review," International Journal of Production Economics, Elsevier, vol. 143(2), pages 499-508.
    4. Kuznetsova, Elizaveta & Li, Yan-Fu & Ruiz, Carlos & Zio, Enrico & Ault, Graham & Bell, Keith, 2013. "Reinforcement learning for microgrid energy management," Energy, Elsevier, vol. 59(C), pages 133-146.
    5. Giannoccaro, Ilaria & Pontrandolfo, Pierpaolo, 2002. "Inventory management in supply chains: a reinforcement learning approach," International Journal of Production Economics, Elsevier, vol. 78(2), pages 153-161, July.
    6. Hu, Qiwei & Boylan, John E. & Chen, Huijing & Labib, Ashraf, 2018. "OR in spare parts management: A review," European Journal of Operational Research, Elsevier, vol. 266(2), pages 395-414.
    7. Stephane R. A. Barde & Soumaya Yacout & Hayong Shin, 2019. "Optimal preventive maintenance policy based on reinforcement learning of a fleet of military trucks," Journal of Intelligent Manufacturing, Springer, vol. 30(1), pages 147-161, January.
    8. Qinming Liu & Ming Dong & Wenyuan Lv & Chunming Ye, 2019. "Manufacturing system maintenance based on dynamic programming model with prognostics information," Journal of Intelligent Manufacturing, Springer, vol. 30(3), pages 1155-1173, March.
    9. Olde Keizer, Minou C.A. & Teunter, Ruud H. & Veldman, Jasper, 2017. "Joint condition-based maintenance and inventory optimization for systems with multiple components," European Journal of Operational Research, Elsevier, vol. 257(1), pages 209-222.
    10. Murthy, D. N. P. & Asgharizadeh, E., 1999. "Optimal decision making in a maintenance service operation," European Journal of Operational Research, Elsevier, vol. 116(2), pages 259-273, July.
    11. Kurz, Julian, 2016. "Capacity planning for a maintenance service provider with advanced information," European Journal of Operational Research, Elsevier, vol. 251(2), pages 466-477.
    12. Jaturonnatee, J. & Murthy, D.N.P. & Boondiskulchok, R., 2006. "Optimal preventive maintenance of leased equipment with corrective minimal repairs," European Journal of Operational Research, Elsevier, vol. 174(1), pages 201-215, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Barlow, E. & Bedford, T. & Revie, M. & Tan, J. & Walls, L., 2021. "A performance-centred approach to optimising maintenance of complex systems," European Journal of Operational Research, Elsevier, vol. 292(2), pages 579-595.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Huber, Sebastian & Spinler, Stefan, 2012. "Pricing of full-service repair contracts," European Journal of Operational Research, Elsevier, vol. 222(1), pages 113-121.
    2. Jackson, Canek & Pascual, Rodrigo & Mac Cawley, Alejandro & Godoy, Sergio, 2023. "Product–service system negotiation in aircraft lease contracts with option of disagreement," Journal of Air Transport Management, Elsevier, vol. 107(C).
    3. Zhang, Xiaohong & Liao, Haitao & Zeng, Jianchao & Shi, Guannan & Zhao, Bing, 2021. "Optimal Condition-based Opportunistic Maintenance and Spare Parts Provisioning for a Two-unit System using a State Space Partitioning Approach," Reliability Engineering and System Safety, Elsevier, vol. 209(C).
    4. Rahimi-Ghahroodi, S. & Al Hanbali, A. & Zijm, W.H.M. & Timmer, J.B., 2019. "Emergency supply contracts for a service provider with limited local resources," Reliability Engineering and System Safety, Elsevier, vol. 189(C), pages 445-460.
    5. Michele Compare & Francesco Di Maio & Enrico Zio & Fausto Carlevaro & Sara Mattafirri, 2016. "Improving scheduled maintenance by missing data reconstruction: A double-loop Monte Carlo approach," Journal of Risk and Reliability, , vol. 230(5), pages 502-511, October.
    6. Correa-Jullian, Camila & López Droguett, Enrique & Cardemil, José Miguel, 2020. "Operation scheduling in a solar thermal system: A reinforcement learning-based framework," Applied Energy, Elsevier, vol. 268(C).
    7. Darghouth, M.N. & Ait-kadi, D. & Chelbi, A., 2017. "Joint optimization of design, warranty and price for products sold with maintenance service contracts," Reliability Engineering and System Safety, Elsevier, vol. 165(C), pages 197-208.
    8. N. Knofius & M. C. Heijden & A. Sleptchenko & W. H. M. Zijm, 2021. "Improving effectiveness of spare parts supply by additive manufacturing as dual sourcing option," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 43(1), pages 189-221, March.
    9. Liu, Xinbao & Yang, Tianji & Pei, Jun & Liao, Haitao & Pohl, Edward A., 2019. "Replacement and inventory control for a multi-customer product service system with decreasing replacement costs," European Journal of Operational Research, Elsevier, vol. 273(2), pages 561-574.
    10. Nguyen, Kim-Anh & Do, Phuc & Grall, Antoine, 2017. "Joint predictive maintenance and inventory strategy for multi-component systems using Birnbaum’s structural importance," Reliability Engineering and System Safety, Elsevier, vol. 168(C), pages 249-261.
    11. Yi Ding & Anatoly Lisnianski & Ilia Frenkel & Lev Khvatskin, 2009. "Optimal corrective maintenance contract planning for aging multi‐state system," Applied Stochastic Models in Business and Industry, John Wiley & Sons, vol. 25(5), pages 612-631, September.
    12. Yan, Tao & Lei, Yaguo & Wang, Biao & Han, Tianyu & Si, Xiaosheng & Li, Naipeng, 2020. "Joint maintenance and spare parts inventory optimization for multi-unit systems considering imperfect maintenance actions," Reliability Engineering and System Safety, Elsevier, vol. 202(C).
    13. Sleptchenko, Andrei & van der Heijden, Matthieu, 2016. "Joint optimization of redundancy level and spare part inventories," Reliability Engineering and System Safety, Elsevier, vol. 153(C), pages 64-74.
    14. Petchrompo, Sanyapong & Parlikad, Ajith Kumar, 2019. "A review of asset management literature on multi-asset systems," Reliability Engineering and System Safety, Elsevier, vol. 181(C), pages 181-201.
    15. Sharafali, Moosa & Tarakci, Hakan & Kulkarni, Shailesh & Razack Shahul Hameed, Raja Abdul, 2019. "Optimal delivery due date for a supplier with an unreliable machine under outsourced maintenance," International Journal of Production Economics, Elsevier, vol. 208(C), pages 53-68.
    16. Zhang, Xiaohong & Zeng, Jianchao, 2017. "Joint optimization of condition-based opportunistic maintenance and spare parts provisioning policy in multiunit systems," European Journal of Operational Research, Elsevier, vol. 262(2), pages 479-498.
    17. Zhu, Mixin & Zhou, Xiaojun, 2022. "Hypergraph-based joint optimization of spare part provision and maintenance scheduling for serial-parallel multi-station manufacturing systems," Reliability Engineering and System Safety, Elsevier, vol. 225(C).
    18. Barlow, E. & Bedford, T. & Revie, M. & Tan, J. & Walls, L., 2021. "A performance-centred approach to optimising maintenance of complex systems," European Journal of Operational Research, Elsevier, vol. 292(2), pages 579-595.
    19. Olde Keizer, Minou C.A. & Flapper, Simme Douwe P. & Teunter, Ruud H., 2017. "Condition-based maintenance policies for systems with multiple dependent components: A review," European Journal of Operational Research, Elsevier, vol. 261(2), pages 405-420.
    20. Shi, Zhenyang & Liu, Shaoxuan, 2020. "Optimal inventory control and design refresh selection in managing part obsolescence," European Journal of Operational Research, Elsevier, vol. 287(1), pages 133-144.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:risrel:v:234:y:2020:i:1:p:52-62. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.