IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v13y2021i23p13016-d686945.html
   My bibliography  Save this article

A Q-Learning Rescheduling Approach to the Flexible Job Shop Problem Combining Energy and Productivity Objectives

Author

Listed:
  • Rami Naimi

    (LS2N UMR CNRS 6004, IUT de Nantes, Nantes University, 2 Avenue du Pr. J. Rouxel, 44470 Carquefou, France)

  • Maroua Nouiri

    (LS2N UMR CNRS 6004, IUT de Nantes, Nantes University, 2 Avenue du Pr. J. Rouxel, 44470 Carquefou, France)

  • Olivier Cardin

    (LS2N UMR CNRS 6004, IUT de Nantes, Nantes University, 2 Avenue du Pr. J. Rouxel, 44470 Carquefou, France)

Abstract

The flexible job shop problem (FJSP) has been studied in recent decades due to its dynamic and uncertain nature. Responding to a system’s perturbation in an intelligent way and with minimum energy consumption variation is an important matter. Fortunately, thanks to the development of artificial intelligence and machine learning, a lot of researchers are using these new techniques to solve the rescheduling problem in a flexible job shop. Reinforcement learning, which is a popular approach in artificial intelligence, is often used in rescheduling. This article presents a Q-learning rescheduling approach to the flexible job shop problem combining energy and productivity objectives in a context of machine failure. First, a genetic algorithm was adopted to generate the initial predictive schedule, and then rescheduling strategies were developed to handle machine failures. As the system should be capable of reacting quickly to unexpected events, a multi-objective Q-learning algorithm is proposed and trained to select the optimal rescheduling methods that minimize the makespan and the energy consumption variation at the same time. This approach was conducted on benchmark instances to evaluate its performance.

Suggested Citation

  • Rami Naimi & Maroua Nouiri & Olivier Cardin, 2021. "A Q-Learning Rescheduling Approach to the Flexible Job Shop Problem Combining Energy and Productivity Objectives," Sustainability, MDPI, vol. 13(23), pages 1-36, November.
    • Handle: RePEc:gam:jsusta:v:13:y:2021:i:23:p:13016-:d:686945
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/13/23/13016/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2071-1050/13/23/13016/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Yu-Fang Wang, 2020. "Adaptive job shop scheduling strategy based on weighted Q-learning algorithm," Journal of Intelligent Manufacturing, Springer, vol. 31(2), pages 417-432, February.
    2. Silviu Raileanu & Florin Anton & Alexandru Iatan & Theodor Borangiu & Silvia Anton & Octavian Morariu, 2017. "Resource scheduling based on energy consumption for sustainable manufacturing," Journal of Intelligent Manufacturing, Springer, vol. 28(7), pages 1519-1530, October.
    3. B. Mihoubi & B. Bouzouia & M. Gaham, 2021. "Reactive scheduling approach for solving a realistic flexible job shop scheduling problem," International Journal of Production Research, Taylor & Francis Journals, vol. 59(19), pages 5790-5808, October.
    4. Sylverin Kemmoe & Damien Lamy & Nikolay Tchernev, 2017. "Job-shop like manufacturing system with variable power threshold and operations with power requirements," International Journal of Production Research, Taylor & Francis Journals, vol. 55(20), pages 6011-6032, October.
    5. Maroua Nouiri & Abdelghani Bekrar & Damien Trentesaux, 2020. "An energy-efficient scheduling and rescheduling method for production and logistics systems†," International Journal of Production Research, Taylor & Francis Journals, vol. 58(11), pages 3263-3283, June.
    6. Liu, Ying & Dong, Haibo & Lohse, Niels & Petrovic, Sanja, 2016. "A multi-objective genetic algorithm for optimisation of energy consumption and shop floor production performance," International Journal of Production Economics, Elsevier, vol. 179(C), pages 259-272.
    7. Masmoudi, Oussama & Delorme, Xavier & Gianessi, Paolo, 2019. "Job-shop scheduling problem with energy consideration," International Journal of Production Economics, Elsevier, vol. 216(C), pages 12-22.
    8. Chuang Wang & Pingyu Jiang, 2018. "Manifold learning based rescheduling decision mechanism for recessive disturbances in RFID-driven job shops," Journal of Intelligent Manufacturing, Springer, vol. 29(7), pages 1485-1500, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. João M. R. C. Fernandes & Seyed Mahdi Homayouni & Dalila B. M. M. Fontes, 2022. "Energy-Efficient Scheduling in Job Shop Manufacturing Systems: A Literature Review," Sustainability, MDPI, vol. 14(10), pages 1-34, May.
    2. Hajo Terbrack & Thorsten Claus & Frank Herrmann, 2021. "Energy-Oriented Production Planning in Industry: A Systematic Literature Review and Classification Scheme," Sustainability, MDPI, vol. 13(23), pages 1-32, December.
    3. Min Dai & Ziwei Zhang & Adriana Giret & Miguel A. Salido, 2019. "An Enhanced Estimation of Distribution Algorithm for Energy-Efficient Job-Shop Scheduling Problems with Transportation Constraints," Sustainability, MDPI, vol. 11(11), pages 1-23, May.
    4. Ivan Ferretti & Matteo Camparada & Lucio Enrico Zavanella, 2022. "Queuing Theory-Based Design Methods for the Definition of Power Requirements in Manufacturing Systems," Energies, MDPI, vol. 15(20), pages 1-14, October.
    5. Park, Myoung-Ju & Ham, Andy, 2022. "Energy-aware flexible job shop scheduling under time-of-use pricing," International Journal of Production Economics, Elsevier, vol. 248(C).
    6. Masmoudi, Oussama & Delorme, Xavier & Gianessi, Paolo, 2019. "Job-shop scheduling problem with energy consideration," International Journal of Production Economics, Elsevier, vol. 216(C), pages 12-22.
    7. Sebastian Mayer & Tobias Classen & Christian Endisch, 2021. "Modular production control using deep reinforcement learning: proximal policy optimization," Journal of Intelligent Manufacturing, Springer, vol. 32(8), pages 2335-2351, December.
    8. Weiwei Cui & Biao Lu, 2020. "A Bi-Objective Approach to Minimize Makespan and Energy Consumption in Flow Shops with Peak Demand Constraint," Sustainability, MDPI, vol. 12(10), pages 1-22, May.
    9. Golpîra, Hêriş, 2020. "Smart Energy-Aware Manufacturing Plant Scheduling under Uncertainty: A Risk-Based Multi-Objective Robust Optimization Approach," Energy, Elsevier, vol. 209(C).
    10. Wichmann, Matthias Gerhard & Johannes, Christoph & Spengler, Thomas Stefan, 2019. "Energy-oriented Lot-Sizing and Scheduling considering energy storages," International Journal of Production Economics, Elsevier, vol. 216(C), pages 204-214.
    11. Xuan Su & Wenquan Dong & Jingyu Lu & Chen Chen & Weixi Ji, 2022. "Dynamic Allocation of Manufacturing Resources in IoT Job Shop Considering Machine State Transfer and Carbon Emission," Sustainability, MDPI, vol. 14(23), pages 1-21, December.
    12. Juan Pablo Usuga Cadavid & Samir Lamouri & Bernard Grabot & Robert Pellerin & Arnaud Fortin, 2020. "Machine learning applied in production planning and control: a state-of-the-art in the era of industry 4.0," Journal of Intelligent Manufacturing, Springer, vol. 31(6), pages 1531-1558, August.
    13. Zakaria Chekoubi & Wajdi Trabelsi & Nathalie Sauer & Ilias Majdouline, 2022. "The Integrated Production-Inventory-Routing Problem with Reverse Logistics and Remanufacturing: A Two-Phase Decomposition Heuristic," Sustainability, MDPI, vol. 14(20), pages 1-30, October.
    14. Seokgi Lee & Mona Issabakhsh & Hyun Woo Jeon & Seong Wook Hwang & Byung Chung, 2020. "Idle time and capacity control for a single machine scheduling problem with dynamic electricity pricing," Operations Management Research, Springer, vol. 13(3), pages 197-217, December.
    15. Annear, Luis Mauricio & Akhavan-Tabatabaei, Raha & Schmid, Verena, 2023. "Dynamic assignment of a multi-skilled workforce in job shops: An approximate dynamic programming approach," European Journal of Operational Research, Elsevier, vol. 306(3), pages 1109-1125.
    16. D.-Y. Kim & J.-W. Park & S. Baek & K.-B. Park & H.-R. Kim & J.-I. Park & H.-S. Kim & B.-B. Kim & H.-Y. Oh & K. Namgung & W. Baek, 2020. "A modular factory testbed for the rapid reconfiguration of manufacturing systems," Journal of Intelligent Manufacturing, Springer, vol. 31(3), pages 661-680, March.
    17. Shun Jia & Shang Wang & Jingxiang Lv & Wei Cai & Na Zhang & Zhongwei Zhang & Shuowei Bai, 2021. "Multi-Objective Optimization of CNC Turning Process Parameters Considering Transient-Steady State Energy Consumption," Sustainability, MDPI, vol. 13(24), pages 1-23, December.
    18. Feng, Yanling & Li, Guo & Sethi, Suresh P., 2018. "A three-layer chromosome genetic algorithm for multi-cell scheduling with flexible routes and machine sharing," International Journal of Production Economics, Elsevier, vol. 196(C), pages 269-283.
    19. Zhongwei Zhang & Lihui Wu & Tao Peng & Shun Jia, 2018. "An Improved Scheduling Approach for Minimizing Total Energy Consumption and Makespan in a Flexible Job Shop Environment," Sustainability, MDPI, vol. 11(1), pages 1-21, December.
    20. Xiangxin An & Guojin Si & Tangbin Xia & Qinming Liu & Yaping Li & Rui Miao, 2022. "Operation and Maintenance Optimization for Manufacturing Systems with Energy Management," Energies, MDPI, vol. 15(19), pages 1-19, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:13:y:2021:i:23:p:13016-:d:686945. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.