IDEAS home Printed from https://ideas.repec.org/a/spr/joinma/v35y2024i2d10.1007_s10845-022-02069-x.html
   My bibliography  Save this article

Robust-stable scheduling in dynamic flow shops based on deep reinforcement learning

Author

Listed:
  • Felix Grumbach

    (Bielefeld University of Applied Sciences)

  • Anna Müller

    (Bielefeld University of Applied Sciences)

  • Pascal Reusch

    (Bielefeld University of Applied Sciences)

  • Sebastian Trojahn

    (Anhalt University of Applied Sciences)

Abstract

This proof-of-concept study provides a novel method for robust-stable scheduling in dynamic flow shops based on deep reinforcement learning (DRL) implemented with OpenAI frameworks. In realistic manufacturing environments, dynamic events endanger baseline schedules, which can require a cost intensive re-scheduling. Extensive research has been done on methods for generating proactive baseline schedules to absorb uncertainties in advance and in balancing the competing metrics of robustness and stability. Recent studies presented exact methods and heuristics based on Monte Carlo experiments (MCE), both of which are very computationally intensive. Furthermore, approaches based on surrogate measures were proposed, which do not explicitly consider uncertainties and robustness metrics. Surprisingly, DRL has not yet been scientifically investigated for generating robust-stable schedules in the proactive stage of production planning. The contribution of this article is a proposal on how DRL can be applied to manipulate operation slack times by stretching or compressing plan durations. The method is demonstrated using different flow shop instances with uncertain processing times, stochastic machine failures and uncertain repair times. Through a computational study, we found that DRL agents achieve about 98% result quality but only take about 2% of the time compared to traditional metaheuristics. This is a promising advantage for the use in real-time environments and supports the idea of improving proactive scheduling methods with machine learning based techniques.

Suggested Citation

  • Felix Grumbach & Anna Müller & Pascal Reusch & Sebastian Trojahn, 2024. "Robust-stable scheduling in dynamic flow shops based on deep reinforcement learning," Journal of Intelligent Manufacturing, Springer, vol. 35(2), pages 667-686, February.
  • Handle: RePEc:spr:joinma:v:35:y:2024:i:2:d:10.1007_s10845-022-02069-x
    DOI: 10.1007/s10845-022-02069-x
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10845-022-02069-x
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s10845-022-02069-x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Vincent F. Yu & Winarno & Achmad Maulidin & A. A. N. Perwira Redi & Shih-Wei Lin & Chao-Lung Yang, 2021. "Simulated Annealing with Restart Strategy for the Path Cover Problem with Time Windows," Mathematics, MDPI, vol. 9(14), pages 1-22, July.
    2. Juan, Angel A. & Faulin, Javier & Grasman, Scott E. & Rabe, Markus & Figueira, Gonçalo, 2015. "A review of simheuristics: Extending metaheuristics to deal with stochastic combinatorial optimization problems," Operations Research Perspectives, Elsevier, vol. 2(C), pages 62-72.
    3. Xiong, Jian & Xing, Li-ning & Chen, Ying-wu, 2013. "Robust scheduling for multi-objective flexible job-shop problems with random machine breakdowns," International Journal of Production Economics, Elsevier, vol. 141(1), pages 112-126.
    4. Shichang Xiao & Shudong Sun & Jionghua (Judy) Jin, 2017. "Surrogate Measures for the Robust Scheduling of Stochastic Job Shop Scheduling Problems," Energies, MDPI, vol. 10(4), pages 1-26, April.
    5. Xichao Su & Wei Han & Yu Wu & Yong Zhang & Jie Liu, 2018. "A Proactive Robust Scheduling Method for Aircraft Carrier Flight Deck Operations with Stochastic Durations," Complexity, Hindawi, vol. 2018, pages 1-38, November.
    6. Taillard, E., 1993. "Benchmarks for basic scheduling problems," European Journal of Operational Research, Elsevier, vol. 64(2), pages 278-285, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Shichang Xiao & Zigao Wu & Hongyan Dui, 2022. "Resilience-Based Surrogate Robustness Measure and Optimization Method for Robust Job-Shop Scheduling," Mathematics, MDPI, vol. 10(21), pages 1-22, October.
    2. Zigao Wu & Shaohua Yu & Tiancheng Li, 2019. "A Meta-Model-Based Multi-Objective Evolutionary Approach to Robust Job Shop Scheduling," Mathematics, MDPI, vol. 7(6), pages 1-19, June.
    3. Abdelhamid Boudjelida, 2019. "On the robustness of joint production and maintenance scheduling in presence of uncertainties," Journal of Intelligent Manufacturing, Springer, vol. 30(4), pages 1515-1530, April.
    4. Noordhoek, Marije & Dullaert, Wout & Lai, David S.W. & de Leeuw, Sander, 2018. "A simulation–optimization approach for a service-constrained multi-echelon distribution network," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 114(C), pages 292-311.
    5. Xiong, Jian & Leus, Roel & Yang, Zhenyu & Abbass, Hussein A., 2016. "Evolutionary multi-objective resource allocation and scheduling in the Chinese navigation satellite system project," European Journal of Operational Research, Elsevier, vol. 251(2), pages 662-675.
    6. Shahaboddin Shamshirband & Mohammad Shojafar & A. Hosseinabadi & Maryam Kardgar & M. Nasir & Rodina Ahmad, 2015. "OSGA: genetic-based open-shop scheduling with consideration of machine maintenance in small and medium enterprises," Annals of Operations Research, Springer, vol. 229(1), pages 743-758, June.
    7. Tao Ren & Yan Zhang & Shuenn-Ren Cheng & Chin-Chia Wu & Meng Zhang & Bo-yu Chang & Xin-yue Wang & Peng Zhao, 2020. "Effective Heuristic Algorithms Solving the Jobshop Scheduling Problem with Release Dates," Mathematics, MDPI, vol. 8(8), pages 1-25, July.
    8. Gueret, Christelle & Jussien, Narendra & Prins, Christian, 2000. "Using intelligent backtracking to improve branch-and-bound methods: An application to Open-Shop problems," European Journal of Operational Research, Elsevier, vol. 127(2), pages 344-354, December.
    9. Angel A. Juan & Helena Ramalhinho-Lourenço & Manuel Mateo & Quim Castellà & Barry B. Barrios, 2012. "ILS-ESP: An efficient, simple, and parameter-free algorithm for solving the permutation flow-shop problem," Economics Working Papers 1319, Department of Economics and Business, Universitat Pompeu Fabra.
    10. Sels, Veronique & Craeymeersch, Kjeld & Vanhoucke, Mario, 2011. "A hybrid single and dual population search procedure for the job shop scheduling problem," European Journal of Operational Research, Elsevier, vol. 215(3), pages 512-523, December.
    11. Brammer, Janis & Lutz, Bernhard & Neumann, Dirk, 2022. "Permutation flow shop scheduling with multiple lines and demand plans using reinforcement learning," European Journal of Operational Research, Elsevier, vol. 299(1), pages 75-86.
    12. Yongxin Lu & Yiping Yuan & Jiarula Yasenjiang & Adilanmu Sitahong & Yongsheng Chao & Yunxuan Wang, 2025. "An Optimized Method for Solving the Green Permutation Flow Shop Scheduling Problem Using a Combination of Deep Reinforcement Learning and Improved Genetic Algorithm," Mathematics, MDPI, vol. 13(4), pages 1-37, February.
    13. Romauch, Martin & Hartl, Richard F., 2017. "Capacity planning for cluster tools in the semiconductor industry," International Journal of Production Economics, Elsevier, vol. 194(C), pages 167-180.
    14. Pan, Quan-Ke & Ruiz, Rubén, 2012. "Local search methods for the flowshop scheduling problem with flowtime minimization," European Journal of Operational Research, Elsevier, vol. 222(1), pages 31-43.
    15. Buscher, Udo & Shen, Liji, 2009. "An integrated tabu search algorithm for the lot streaming problem in job shops," European Journal of Operational Research, Elsevier, vol. 199(2), pages 385-399, December.
    16. Jose L. Andrade-Pineda & David Canca & Pedro L. Gonzalez-R & M. Calle, 2020. "Scheduling a dual-resource flexible job shop with makespan and due date-related criteria," Annals of Operations Research, Springer, vol. 291(1), pages 5-35, August.
    17. Naderi, B. & Zandieh, M., 2014. "Modeling and scheduling no-wait open shop problems," International Journal of Production Economics, Elsevier, vol. 158(C), pages 256-266.
    18. Tseng, Lin-Yu & Lin, Ya-Tai, 2009. "A hybrid genetic local search algorithm for the permutation flowshop scheduling problem," European Journal of Operational Research, Elsevier, vol. 198(1), pages 84-92, October.
    19. Gerardo Minella & Rubén Ruiz & Michele Ciavotta, 2008. "A Review and Evaluation of Multiobjective Algorithms for the Flowshop Scheduling Problem," INFORMS Journal on Computing, INFORMS, vol. 20(3), pages 451-471, August.
    20. Pempera, Jaroslaw & Smutnicki, Czeslaw, 2018. "Open shop cyclic scheduling," European Journal of Operational Research, Elsevier, vol. 269(2), pages 773-781.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:joinma:v:35:y:2024:i:2:d:10.1007_s10845-022-02069-x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.