IDEAS home Printed from https://ideas.repec.org/a/spr/joinma/v35y2024i2d10.1007_s10845-022-02069-x.html
   My bibliography  Save this article

Robust-stable scheduling in dynamic flow shops based on deep reinforcement learning

Author

Listed:
  • Felix Grumbach

    (Bielefeld University of Applied Sciences)

  • Anna Müller

    (Bielefeld University of Applied Sciences)

  • Pascal Reusch

    (Bielefeld University of Applied Sciences)

  • Sebastian Trojahn

    (Anhalt University of Applied Sciences)

Abstract

This proof-of-concept study provides a novel method for robust-stable scheduling in dynamic flow shops based on deep reinforcement learning (DRL) implemented with OpenAI frameworks. In realistic manufacturing environments, dynamic events endanger baseline schedules, which can require a cost intensive re-scheduling. Extensive research has been done on methods for generating proactive baseline schedules to absorb uncertainties in advance and in balancing the competing metrics of robustness and stability. Recent studies presented exact methods and heuristics based on Monte Carlo experiments (MCE), both of which are very computationally intensive. Furthermore, approaches based on surrogate measures were proposed, which do not explicitly consider uncertainties and robustness metrics. Surprisingly, DRL has not yet been scientifically investigated for generating robust-stable schedules in the proactive stage of production planning. The contribution of this article is a proposal on how DRL can be applied to manipulate operation slack times by stretching or compressing plan durations. The method is demonstrated using different flow shop instances with uncertain processing times, stochastic machine failures and uncertain repair times. Through a computational study, we found that DRL agents achieve about 98% result quality but only take about 2% of the time compared to traditional metaheuristics. This is a promising advantage for the use in real-time environments and supports the idea of improving proactive scheduling methods with machine learning based techniques.

Suggested Citation

  • Felix Grumbach & Anna Müller & Pascal Reusch & Sebastian Trojahn, 2024. "Robust-stable scheduling in dynamic flow shops based on deep reinforcement learning," Journal of Intelligent Manufacturing, Springer, vol. 35(2), pages 667-686, February.
  • Handle: RePEc:spr:joinma:v:35:y:2024:i:2:d:10.1007_s10845-022-02069-x
    DOI: 10.1007/s10845-022-02069-x
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10845-022-02069-x
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s10845-022-02069-x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Vincent F. Yu & Winarno & Achmad Maulidin & A. A. N. Perwira Redi & Shih-Wei Lin & Chao-Lung Yang, 2021. "Simulated Annealing with Restart Strategy for the Path Cover Problem with Time Windows," Mathematics, MDPI, vol. 9(14), pages 1-22, July.
    2. Juan, Angel A. & Faulin, Javier & Grasman, Scott E. & Rabe, Markus & Figueira, Gonçalo, 2015. "A review of simheuristics: Extending metaheuristics to deal with stochastic combinatorial optimization problems," Operations Research Perspectives, Elsevier, vol. 2(C), pages 62-72.
    3. Xiong, Jian & Xing, Li-ning & Chen, Ying-wu, 2013. "Robust scheduling for multi-objective flexible job-shop problems with random machine breakdowns," International Journal of Production Economics, Elsevier, vol. 141(1), pages 112-126.
    4. Shichang Xiao & Shudong Sun & Jionghua (Judy) Jin, 2017. "Surrogate Measures for the Robust Scheduling of Stochastic Job Shop Scheduling Problems," Energies, MDPI, vol. 10(4), pages 1-26, April.
    5. Xichao Su & Wei Han & Yu Wu & Yong Zhang & Jie Liu, 2018. "A Proactive Robust Scheduling Method for Aircraft Carrier Flight Deck Operations with Stochastic Durations," Complexity, Hindawi, vol. 2018, pages 1-38, November.
    6. Taillard, E., 1993. "Benchmarks for basic scheduling problems," European Journal of Operational Research, Elsevier, vol. 64(2), pages 278-285, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zigao Wu & Shaohua Yu & Tiancheng Li, 2019. "A Meta-Model-Based Multi-Objective Evolutionary Approach to Robust Job Shop Scheduling," Mathematics, MDPI, vol. 7(6), pages 1-19, June.
    2. Abdelhamid Boudjelida, 2019. "On the robustness of joint production and maintenance scheduling in presence of uncertainties," Journal of Intelligent Manufacturing, Springer, vol. 30(4), pages 1515-1530, April.
    3. Shichang Xiao & Zigao Wu & Hongyan Dui, 2022. "Resilience-Based Surrogate Robustness Measure and Optimization Method for Robust Job-Shop Scheduling," Mathematics, MDPI, vol. 10(21), pages 1-22, October.
    4. Noordhoek, Marije & Dullaert, Wout & Lai, David S.W. & de Leeuw, Sander, 2018. "A simulation–optimization approach for a service-constrained multi-echelon distribution network," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 114(C), pages 292-311.
    5. Xiong, Jian & Leus, Roel & Yang, Zhenyu & Abbass, Hussein A., 2016. "Evolutionary multi-objective resource allocation and scheduling in the Chinese navigation satellite system project," European Journal of Operational Research, Elsevier, vol. 251(2), pages 662-675.
    6. Shahaboddin Shamshirband & Mohammad Shojafar & A. Hosseinabadi & Maryam Kardgar & M. Nasir & Rodina Ahmad, 2015. "OSGA: genetic-based open-shop scheduling with consideration of machine maintenance in small and medium enterprises," Annals of Operations Research, Springer, vol. 229(1), pages 743-758, June.
    7. Sels, Veronique & Craeymeersch, Kjeld & Vanhoucke, Mario, 2011. "A hybrid single and dual population search procedure for the job shop scheduling problem," European Journal of Operational Research, Elsevier, vol. 215(3), pages 512-523, December.
    8. Romauch, Martin & Hartl, Richard F., 2017. "Capacity planning for cluster tools in the semiconductor industry," International Journal of Production Economics, Elsevier, vol. 194(C), pages 167-180.
    9. Pempera, Jaroslaw & Smutnicki, Czeslaw, 2018. "Open shop cyclic scheduling," European Journal of Operational Research, Elsevier, vol. 269(2), pages 773-781.
    10. Shahvari, Omid & Logendran, Rasaratnam, 2016. "Hybrid flow shop batching and scheduling with a bi-criteria objective," International Journal of Production Economics, Elsevier, vol. 179(C), pages 239-258.
    11. Taejong Joo & Hyunyoung Jun & Dongmin Shin, 2022. "Task Allocation in Human–Machine Manufacturing Systems Using Deep Reinforcement Learning," Sustainability, MDPI, vol. 14(4), pages 1-18, February.
    12. Fleming, Christopher L. & Griffis, Stanley E. & Bell, John E., 2013. "The effects of triangle inequality on the vehicle routing problem," European Journal of Operational Research, Elsevier, vol. 224(1), pages 1-7.
    13. Jean-Paul Watson & Laura Barbulescu & L. Darrell Whitley & Adele E. Howe, 2002. "Contrasting Structured and Random Permutation Flow-Shop Scheduling Problems: Search-Space Topology and Algorithm Performance," INFORMS Journal on Computing, INFORMS, vol. 14(2), pages 98-123, May.
    14. Nouha Nouri & Talel Ladhari, 2018. "Evolutionary multiobjective optimization for the multi-machine flow shop scheduling problem under blocking," Annals of Operations Research, Springer, vol. 267(1), pages 413-430, August.
    15. Liaw, Ching-Fang, 2000. "A hybrid genetic algorithm for the open shop scheduling problem," European Journal of Operational Research, Elsevier, vol. 124(1), pages 28-42, July.
    16. Pflughoeft, K. A. & Hutchinson, G. K. & Nazareth, D. L., 1996. "Intelligent decision support for flexible manufacturing: Design and implementation of a knowledge-based simulator," Omega, Elsevier, vol. 24(3), pages 347-360, June.
    17. Pan, Quan-Ke & Gao, Liang & Li, Xin-Yu & Gao, Kai-Zhou, 2017. "Effective metaheuristics for scheduling a hybrid flowshop with sequence-dependent setup times," Applied Mathematics and Computation, Elsevier, vol. 303(C), pages 89-112.
    18. Perez-Gonzalez, Paz & Framinan, Jose M., 2024. "A review and classification on distributed permutation flowshop scheduling problems," European Journal of Operational Research, Elsevier, vol. 312(1), pages 1-21.
    19. Liu, Jiyin & Reeves, Colin R, 2001. "Constructive and composite heuristic solutions to the P//[summation operator]Ci scheduling problem," European Journal of Operational Research, Elsevier, vol. 132(2), pages 439-452, July.
    20. Zeynep Adak & Mahmure Övül Arıoğlu Akan & Serol Bulkan, 0. "Multiprocessor open shop problem: literature review and future directions," Journal of Combinatorial Optimization, Springer, vol. 0, pages 1-23.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:joinma:v:35:y:2024:i:2:d:10.1007_s10845-022-02069-x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.