IDEAS home Printed from https://ideas.repec.org/a/sae/risrel/v234y2020i1p52-62.html

A reinforcement learning approach to optimal part flow management for gas turbine maintenance

Author

Listed:
  • Michele Compare
  • Luca Bellani
  • Enrico Cobelli
  • Enrico Zio
  • Francesco Annunziata
  • Fausto Carlevaro
  • Marzia Sepe

Abstract

We consider the maintenance process of gas turbines used in the Oil and Gas industry: the capital parts are first removed from the gas turbines and replaced by parts of the same type taken from the warehouse; then, they are repaired at the workshop and returned to the warehouse for use in future maintenance events. Experience-based rules are used to manage the flow of the parts for a profitable gas turbine operation. In this article, we formalize the part flow management as a sequential decision problem and propose reinforcement learning for its solution. An application to a scaled-down case study derived from real industrial practice shows that reinforcement learning can find policies outperforming those based on experience-based rules.

Suggested Citation

  • Michele Compare & Luca Bellani & Enrico Cobelli & Enrico Zio & Francesco Annunziata & Fausto Carlevaro & Marzia Sepe, 2020. "A reinforcement learning approach to optimal part flow management for gas turbine maintenance," Journal of Risk and Reliability, , vol. 234(1), pages 52-62, February.
    • Handle: RePEc:sae:risrel:v:234:y:2020:i:1:p:52-62
    DOI: 10.1177/1748006X19869750
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/1748006X19869750
    Download Restriction: no
    File URL: https://libkey.io/10.1177/1748006X19869750?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Jin, Tongdan & Tian, Zhigang & Xie, Min, 2015. "A game-theoretical approach for optimizing maintenance, spares and service capacity in performance contracting," International Journal of Production Economics, Elsevier, vol. 161(C), pages 31-43.
    2. Wang, Wenbin, 2010. "A model for maintenance service contract design, negotiation and optimization," European Journal of Operational Research, Elsevier, vol. 201(1), pages 239-246, February.
    3. Hu, Qiwei & Boylan, John E. & Chen, Huijing & Labib, Ashraf, 2018. "OR in spare parts management: A review," European Journal of Operational Research, Elsevier, vol. 266(2), pages 395-414.
    4. Qinming Liu & Ming Dong & Wenyuan Lv & Chunming Ye, 2019. "Manufacturing system maintenance based on dynamic programming model with prognostics information," Journal of Intelligent Manufacturing, Springer, vol. 30(3), pages 1155-1173, March.
    5. Olde Keizer, Minou C.A. & Teunter, Ruud H. & Veldman, Jasper, 2017. "Joint condition-based maintenance and inventory optimization for systems with multiple components," European Journal of Operational Research, Elsevier, vol. 257(1), pages 209-222.
    6. Murthy, D. N. P. & Asgharizadeh, E., 1999. "Optimal decision making in a maintenance service operation," European Journal of Operational Research, Elsevier, vol. 116(2), pages 259-273, July.
    7. Kurz, Julian, 2016. "Capacity planning for a maintenance service provider with advanced information," European Journal of Operational Research, Elsevier, vol. 251(2), pages 466-477.
    8. Van Horenbeek, Adriaan & Buré, Jasmine & Cattrysse, Dirk & Pintelon, Liliane & Vansteenwegen, Pieter, 2013. "Joint maintenance and inventory optimization systems: A review," International Journal of Production Economics, Elsevier, vol. 143(2), pages 499-508.
    9. Kuznetsova, Elizaveta & Li, Yan-Fu & Ruiz, Carlos & Zio, Enrico & Ault, Graham & Bell, Keith, 2013. "Reinforcement learning for microgrid energy management," Energy, Elsevier, vol. 59(C), pages 133-146.
    10. Giannoccaro, Ilaria & Pontrandolfo, Pierpaolo, 2002. "Inventory management in supply chains: a reinforcement learning approach," International Journal of Production Economics, Elsevier, vol. 78(2), pages 153-161, July.
    11. Stephane R. A. Barde & Soumaya Yacout & Hayong Shin, 2019. "Optimal preventive maintenance policy based on reinforcement learning of a fleet of military trucks," Journal of Intelligent Manufacturing, Springer, vol. 30(1), pages 147-161, January.
    12. Jaturonnatee, J. & Murthy, D.N.P. & Boondiskulchok, R., 2006. "Optimal preventive maintenance of leased equipment with corrective minimal repairs," European Journal of Operational Research, Elsevier, vol. 174(1), pages 201-215, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Barlow, E. & Bedford, T. & Revie, M. & Tan, J. & Walls, L., 2021. "A performance-centred approach to optimising maintenance of complex systems," European Journal of Operational Research, Elsevier, vol. 292(2), pages 579-595.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Huber, Sebastian & Spinler, Stefan, 2012. "Pricing of full-service repair contracts," European Journal of Operational Research, Elsevier, vol. 222(1), pages 113-121.
    2. Rozas, Heraldo & Xie, Weijun & Gebraeel, Nagi & Robinson, Stephen, 2026. "Data-driven joint optimization of maintenance and spare parts provisioning: A distributionally robust approach," European Journal of Operational Research, Elsevier, vol. 328(1), pages 122-136.
    3. Jackson, Canek & Pascual, Rodrigo & Mac Cawley, Alejandro & Godoy, Sergio, 2023. "Product–service system negotiation in aircraft lease contracts with option of disagreement," Journal of Air Transport Management, Elsevier, vol. 107(C).
    4. Wang, Yukun & Gao, Weizheng & Li, Xiaopeng & Liu, Yiliu, 2024. "Joint optimization of performance-based contracting, condition-based maintenance and spare parts inventory for degrading production systems," Reliability Engineering and System Safety, Elsevier, vol. 243(C).
    5. Zhang, Xiaohong & Liao, Haitao & Zeng, Jianchao & Shi, Guannan & Zhao, Bing, 2021. "Optimal Condition-based Opportunistic Maintenance and Spare Parts Provisioning for a Two-unit System using a State Space Partitioning Approach," Reliability Engineering and System Safety, Elsevier, vol. 209(C).
    6. Rahimi-Ghahroodi, S. & Al Hanbali, A. & Zijm, W.H.M. & Timmer, J.B., 2019. "Emergency supply contracts for a service provider with limited local resources," Reliability Engineering and System Safety, Elsevier, vol. 189(C), pages 445-460.
    7. Jing Zhang & Xian Zhao & Yanbo Song, 2021. "Joint optimization of reliability design and spares inventory for a use-oriented product-service system based on the electric vehicle charging station," Journal of Risk and Reliability, , vol. 235(4), pages 676-689, August.
    8. Jong-Jang Lin & Yi-Kuei Lin & Louis Cheng-Lu Yeng & Ruey-Huei Yeh, 2025. "An outcome-based maintenance contracting model by considering multi-criteria and risk equilibrium," Annals of Operations Research, Springer, vol. 349(1), pages 47-66, June.
    9. Michele Compare & Francesco Di Maio & Enrico Zio & Fausto Carlevaro & Sara Mattafirri, 2016. "Improving scheduled maintenance by missing data reconstruction: A double-loop Monte Carlo approach," Journal of Risk and Reliability, , vol. 230(5), pages 502-511, October.
    10. Correa-Jullian, Camila & López Droguett, Enrique & Cardemil, José Miguel, 2020. "Operation scheduling in a solar thermal system: A reinforcement learning-based framework," Applied Energy, Elsevier, vol. 268(C).
    11. Darghouth, M.N. & Ait-kadi, D. & Chelbi, A., 2017. "Joint optimization of design, warranty and price for products sold with maintenance service contracts," Reliability Engineering and System Safety, Elsevier, vol. 165(C), pages 197-208.
    12. Kouki, Chaaben & Drent, Melvin & Babai, M. Zied & Drent, Collin, 2025. "Dedicated maintenance and repair shop control for spare parts networks," Other publications TiSEM 8609700c-5ae3-4e17-b2f8-f, Tilburg University, School of Economics and Management.
    13. N. Knofius & M. C. Heijden & A. Sleptchenko & W. H. M. Zijm, 2021. "Improving effectiveness of spare parts supply by additive manufacturing as dual sourcing option," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 43(1), pages 189-221, March.
    14. Liu, Xinbao & Yang, Tianji & Pei, Jun & Liao, Haitao & Pohl, Edward A., 2019. "Replacement and inventory control for a multi-customer product service system with decreasing replacement costs," European Journal of Operational Research, Elsevier, vol. 273(2), pages 561-574.
    15. Jin, Tongdan, 2025. "Contracting for system availability with zero ownership cost via redundancy, maintenance and repairable inventory," International Journal of Production Economics, Elsevier, vol. 281(C).
    16. Nguyen, Kim-Anh & Do, Phuc & Grall, Antoine, 2017. "Joint predictive maintenance and inventory strategy for multi-component systems using Birnbaum’s structural importance," Reliability Engineering and System Safety, Elsevier, vol. 168(C), pages 249-261.
    17. Shuyuan Gan & Xinzhou Zhang & Lan Chen, 2022. "Maintenance optimization of a production system considering defect prevention and spare parts ordering," Journal of Risk and Reliability, , vol. 236(5), pages 893-906, October.
    18. Arts, Joachim & Boute, Robert N. & Loeys, Stijn & van Staden, Heletjé E., 2025. "Fifty years of maintenance optimization: Reflections and perspectives," European Journal of Operational Research, Elsevier, vol. 322(3), pages 725-739.
    19. Yi Ding & Anatoly Lisnianski & Ilia Frenkel & Lev Khvatskin, 2009. "Optimal corrective maintenance contract planning for aging multi‐state system," Applied Stochastic Models in Business and Industry, John Wiley & Sons, vol. 25(5), pages 612-631, September.
    20. Yan, Tao & Lei, Yaguo & Wang, Biao & Han, Tianyu & Si, Xiaosheng & Li, Naipeng, 2020. "Joint maintenance and spare parts inventory optimization for multi-unit systems considering imperfect maintenance actions," Reliability Engineering and System Safety, Elsevier, vol. 202(C).

    More about this item

    Keywords

    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:risrel:v:234:y:2020:i:1:p:52-62. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.