IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v15y2023i4p2920-d1059335.html
   My bibliography  Save this article

Deep Reinforcement Learning-Based Scheduler on Parallel Dedicated Machine Scheduling Problem towards Minimizing Total Tardiness

Author

Listed:
  • Donghun Lee

    (Department of Industrial and Management Engineering, Incheon National University, Incheon 22012, Republic of Korea)

  • Hyeongwon Kang

    (Department of Industrial and Management Engineering, Incheon National University, Incheon 22012, Republic of Korea)

  • Dongjin Lee

    (Department of Industrial and Management Engineering, Incheon National University, Incheon 22012, Republic of Korea)

  • Jeonwoo Lee

    (Department of Industrial and Management Engineering, Incheon National University, Incheon 22012, Republic of Korea)

  • Kwanho Kim

    (Department of Industrial and Management Engineering, Incheon National University, Incheon 22012, Republic of Korea)

Abstract

This study considers a parallel dedicated machine scheduling problem towards minimizing the total tardiness of allocated jobs on machines. In addition, this problem comes under the category of NP-hard. Unlike classical parallel machine scheduling, a job is processed by only one of the dedicated machines according to its job type defined in advance, and a machine is able to process at most one job at a time. To obtain a high-quality schedule in terms of total tardiness for the considered scheduling problem, we suggest a machine scheduler based on double deep Q-learning. In the training phase, the considered scheduling problem is redesigned to fit into the reinforcement learning framework and suggest the concepts of state, action, and reward to understand the occurrences of setup, tardiness, and the statuses of allocated job types. The proposed scheduler, repeatedly finds better Q-values towards minimizing tardiness of allocated jobs by updating the weights in a neural network. Then, the scheduling performances of the proposed scheduler are evaluated by comparing it with the conventional ones. The results show that the proposed scheduler outperforms the conventional ones. In particular, for two datasets presenting extra-large scheduling problems, our model performs better compared to existing genetic algorithm by 12.32% and 29.69%.

Suggested Citation

  • Donghun Lee & Hyeongwon Kang & Dongjin Lee & Jeonwoo Lee & Kwanho Kim, 2023. "Deep Reinforcement Learning-Based Scheduler on Parallel Dedicated Machine Scheduling Problem towards Minimizing Total Tardiness," Sustainability, MDPI, vol. 15(4), pages 1-14, February.
  • Handle: RePEc:gam:jsusta:v:15:y:2023:i:4:p:2920-:d:1059335
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/15/4/2920/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/2071-1050/15/4/2920/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Biskup, Dirk & Herrmann, Jan & Gupta, Jatinder N.D., 2008. "Scheduling identical parallel machines to minimize total tardiness," International Journal of Production Economics, Elsevier, vol. 115(1), pages 134-142, September.
    2. Ruiz, Ruben & Stutzle, Thomas, 2008. "An Iterated Greedy heuristic for the sequence dependent setup times flowshop problem with makespan and weighted tardiness objectives," European Journal of Operational Research, Elsevier, vol. 187(3), pages 1143-1159, June.
    3. Lee, Young Hoon & Pinedo, Michael, 1997. "Scheduling jobs on parallel machines with sequence-dependent setup times," European Journal of Operational Research, Elsevier, vol. 100(3), pages 464-474, August.
    4. Daming Shi & Wenhui Fan & Yingying Xiao & Tingyu Lin & Chi Xing, 2020. "Intelligent scheduling of discrete automated production line via deep reinforcement learning," International Journal of Production Research, Taylor & Francis Journals, vol. 58(11), pages 3362-3380, June.
    5. Ari P. J. Vepsalainen & Thomas E. Morton, 1987. "Priority Rules for Job Shops with Weighted Tardiness Costs," Management Science, INFORMS, vol. 33(8), pages 1035-1047, August.
    6. Volodymyr Mnih & Koray Kavukcuoglu & David Silver & Andrei A. Rusu & Joel Veness & Marc G. Bellemare & Alex Graves & Martin Riedmiller & Andreas K. Fidjeland & Georg Ostrovski & Stig Petersen & Charle, 2015. "Human-level control through deep reinforcement learning," Nature, Nature, vol. 518(7540), pages 529-533, February.
    7. Jun-Ho Lee & Hoon Jang, 2019. "Uniform Parallel Machine Scheduling with Dedicated Machines, Job Splitting and Setup Resources," Sustainability, MDPI, vol. 11(24), pages 1-23, December.
    8. Peng Zhang & Youlong Lv & Jie Zhang, 2018. "An improved imperialist competitive algorithm based photolithography machines scheduling," International Journal of Production Research, Taylor & Francis Journals, vol. 56(3), pages 1017-1029, February.
    9. Shim, Sang-Oh & Kim, Yeong-Dae, 2007. "Scheduling on parallel identical machines to minimize total tardiness," European Journal of Operational Research, Elsevier, vol. 177(1), pages 135-146, February.
    10. Yong Jae Kim & Jae Won Jang & David S. Kim & Byung Soo Kim, 2022. "Batch loading and scheduling problem with processing time deterioration and rate-modifying activities," International Journal of Production Research, Taylor & Francis Journals, vol. 60(5), pages 1600-1620, March.
    11. Gaia Nicosia & Andrea Pacifici, 2017. "Scheduling assembly tasks with caterpillar precedence constraints on dedicated machines," International Journal of Production Research, Taylor & Francis Journals, vol. 55(6), pages 1680-1691, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Söhnke Maecker & Liji Shen, 2020. "Solving parallel machine problems with delivery times and tardiness objectives," Annals of Operations Research, Springer, vol. 285(1), pages 315-334, February.
    2. Arthur Kramer & Anand Subramanian, 2019. "A unified heuristic and an annotated bibliography for a large class of earliness–tardiness scheduling problems," Journal of Scheduling, Springer, vol. 22(1), pages 21-57, February.
    3. Alidaee, Bahram & Kochenberger, Gary A. & Amini, Mohammad M., 2001. "Greedy solutions of selection and ordering problems," European Journal of Operational Research, Elsevier, vol. 134(1), pages 203-215, October.
    4. Brammer, Janis & Lutz, Bernhard & Neumann, Dirk, 2022. "Permutation flow shop scheduling with multiple lines and demand plans using reinforcement learning," European Journal of Operational Research, Elsevier, vol. 299(1), pages 75-86.
    5. Daniel Schubert & André Scholz & Gerhard Wäscher, 2017. "Integrated Order Picking and Vehicle Routing with Due Dates," FEMM Working Papers 170007, Otto-von-Guericke University Magdeburg, Faculty of Economics and Management.
    6. Daniel Schubert & André Scholz & Gerhard Wäscher, 2018. "Integrated order picking and vehicle routing with due dates," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 40(4), pages 1109-1139, October.
    7. Xi, Yue & Jang, Jaejin, 2012. "Scheduling jobs on identical parallel machines with unequal future ready time and sequence dependent setup: An experimental study," International Journal of Production Economics, Elsevier, vol. 137(1), pages 1-10.
    8. Mensendiek, Arne & Gupta, Jatinder N.D. & Herrmann, Jan, 2015. "Scheduling identical parallel machines with fixed delivery dates to minimize total tardiness," European Journal of Operational Research, Elsevier, vol. 243(2), pages 514-522.
    9. Maria Raquel C. Costa & Jorge M. S. Valente & Jeffrey E. Schaller, 2020. "Efficient procedures for the weighted squared tardiness permutation flowshop scheduling problem," Flexible Services and Manufacturing Journal, Springer, vol. 32(3), pages 487-522, September.
    10. Shim, Sang-Oh & Kim, Yeong-Dae, 2007. "Scheduling on parallel identical machines to minimize total tardiness," European Journal of Operational Research, Elsevier, vol. 177(1), pages 135-146, February.
    11. S-O Shim & Y-D Kim, 2007. "Minimizing total tardiness in an unrelated parallel-machine scheduling problem," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 58(3), pages 346-354, March.
    12. Huiqiao Su & Michael Pinedo & Guohua Wan, 2017. "Parallel machine scheduling with eligibility constraints: A composite dispatching rule to minimize total weighted tardiness," Naval Research Logistics (NRL), John Wiley & Sons, vol. 64(3), pages 249-267, April.
    13. Mohammed Alnahhal & Nikola Gjeldum & Bashir Salah, 2023. "Optimal Scheduling of Rainwater Collection Vehicles: Mixed Integer Programming and Genetic Algorithms," Sustainability, MDPI, vol. 15(12), pages 1-18, June.
    14. Tulika Saha & Sriparna Saha & Pushpak Bhattacharyya, 2020. "Towards sentiment aided dialogue policy learning for multi-intent conversations using hierarchical reinforcement learning," PLOS ONE, Public Library of Science, vol. 15(7), pages 1-28, July.
    15. Mahmoud Mahfouz & Angelos Filos & Cyrine Chtourou & Joshua Lockhart & Samuel Assefa & Manuela Veloso & Danilo Mandic & Tucker Balch, 2019. "On the Importance of Opponent Modeling in Auction Markets," Papers 1911.12816, arXiv.org.
    16. Imen Azzouz & Wiem Fekih Hassen, 2023. "Optimization of Electric Vehicles Charging Scheduling Based on Deep Reinforcement Learning: A Decentralized Approach," Energies, MDPI, vol. 16(24), pages 1-18, December.
    17. Jacob W. Crandall & Mayada Oudah & Tennom & Fatimah Ishowo-Oloko & Sherief Abdallah & Jean-François Bonnefon & Manuel Cebrian & Azim Shariff & Michael A. Goodrich & Iyad Rahwan, 2018. "Cooperating with machines," Nature Communications, Nature, vol. 9(1), pages 1-12, December.
      • Abdallah, Sherief & Bonnefon, Jean-François & Cebrian, Manuel & Crandall, Jacob W. & Ishowo-Oloko, Fatimah & Oudah, Mayada & Rahwan, Iyad & Shariff, Azim & Tennom,, 2017. "Cooperating with Machines," TSE Working Papers 17-806, Toulouse School of Economics (TSE).
      • Abdallah, Sherief & Bonnefon, Jean-François & Cebrian, Manuel & Crandall, Jacob W. & Ishowo-Oloko, Fatimah & Oudah, Mayada & Rahwan, Iyad & Shariff, Azim & Tennom,, 2017. "Cooperating with Machines," IAST Working Papers 17-68, Institute for Advanced Study in Toulouse (IAST).
      • Jacob Crandall & Mayada Oudah & Fatimah Ishowo-Oloko Tennom & Fatimah Ishowo-Oloko & Sherief Abdallah & Jean-François Bonnefon & Manuel Cebrian & Azim Shariff & Michael Goodrich & Iyad Rahwan, 2018. "Cooperating with machines," Post-Print hal-01897802, HAL.
    18. Sun, Alexander Y., 2020. "Optimal carbon storage reservoir management through deep reinforcement learning," Applied Energy, Elsevier, vol. 278(C).
    19. Yassine Chemingui & Adel Gastli & Omar Ellabban, 2020. "Reinforcement Learning-Based School Energy Management System," Energies, MDPI, vol. 13(23), pages 1-21, December.
    20. Jianxin Fang & Brenda Cheang & Andrew Lim, 2023. "Problems and Solution Methods of Machine Scheduling in Semiconductor Manufacturing Operations: A Survey," Sustainability, MDPI, vol. 15(17), pages 1-44, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:15:y:2023:i:4:p:2920-:d:1059335. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.