IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v304y2023i3p1296-1330.html
   My bibliography  Save this article

Learning to select operators in meta-heuristics: An integration of Q-learning into the iterated greedy algorithm for the permutation flowshop scheduling problem

Author

Listed:
  • Karimi-Mamaghan, Maryam
  • Mohammadi, Mehrdad
  • Pasdeloup, Bastien
  • Meyer, Patrick

Abstract

This paper aims at integrating machine learning techniques into meta-heuristics for solving combinatorial optimization problems. Specifically, our study develops a novel efficient iterated greedy algorithm based on reinforcement learning. The main novelty of the proposed algorithm is its new perturbation mechanism, which incorporates Q-learning to select appropriate perturbation operators during the search process. Through an application to the permutation flowshop scheduling problem, comprehensive computational experiments are conducted on a wide range of benchmark instances to evaluate the performance of the proposed algorithm. This evaluation is done against non-learning versions of the iterated greedy algorithm and seven state-of-the-art algorithms from the literature. The experimental results and statistical analyses show the better performance of the proposed algorithm in terms of optimality gaps, convergence rate, and computational overhead.

Suggested Citation

  • Karimi-Mamaghan, Maryam & Mohammadi, Mehrdad & Pasdeloup, Bastien & Meyer, Patrick, 2023. "Learning to select operators in meta-heuristics: An integration of Q-learning into the iterated greedy algorithm for the permutation flowshop scheduling problem," European Journal of Operational Research, Elsevier, vol. 304(3), pages 1296-1330.
  • Handle: RePEc:eee:ejores:v:304:y:2023:i:3:p:1296-1330
    DOI: 10.1016/j.ejor.2022.03.054
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221722002788
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ejor.2022.03.054?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Pan, Quan-Ke & Ruiz, Rubén, 2012. "Local search methods for the flowshop scheduling problem with flowtime minimization," European Journal of Operational Research, Elsevier, vol. 222(1), pages 31-43.
    2. Mosadegh, H. & Fatemi Ghomi, S.M.T. & Süer, G.A., 2020. "Stochastic mixed-model assembly line sequencing problem: Mathematical modeling and Q-learning based simulated annealing hyper-heuristics," European Journal of Operational Research, Elsevier, vol. 282(2), pages 530-544.
    3. Fernandez-Viagas, Victor & Ruiz, Rubén & Framinan, Jose M., 2017. "A new vision of approximate methods for the permutation flowshop to minimise makespan: State-of-the-art and computational evaluation," European Journal of Operational Research, Elsevier, vol. 257(3), pages 707-721.
    4. Pagnozzi, Federico & Stützle, Thomas, 2019. "Automatic design of hybrid stochastic local search algorithms for permutation flowshop problems," European Journal of Operational Research, Elsevier, vol. 276(2), pages 409-421.
    5. Wawrzyniak, Jakub & Drozdowski, Maciej & Sanlaville, Éric, 2020. "Selecting algorithms for large berth allocation problems," European Journal of Operational Research, Elsevier, vol. 283(3), pages 844-862.
    6. Karimi-Mamaghan, Maryam & Mohammadi, Mehrdad & Pirayesh, Amir & Karimi-Mamaghan, Amir Mohammad & Irani, Hassan, 2020. "Hub-and-spoke network design under congestion: A learning based metaheuristic," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 142(C).
    7. Mohammadi, Mehrdad & Jula, Payman & Tavakkoli-Moghaddam, Reza, 2019. "Reliable single-allocation hub location problem with disruptions," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 123(C), pages 90-120.
    8. Ömür Tosun & M.K. Marichelvam, 2016. "Hybrid bat algorithm for flow shop scheduling problems," International Journal of Mathematics in Operational Research, Inderscience Enterprises Ltd, vol. 9(1), pages 125-138.
    9. Vallada, Eva & Ruiz, Rubén & Framinan, Jose M., 2015. "New hard benchmark for flowshop scheduling problems minimising makespan," European Journal of Operational Research, Elsevier, vol. 240(3), pages 666-677.
    10. Taillard, E., 1990. "Some efficient heuristic methods for the flow shop sequencing problem," European Journal of Operational Research, Elsevier, vol. 47(1), pages 65-74, July.
    11. Taillard, E., 1993. "Benchmarks for basic scheduling problems," European Journal of Operational Research, Elsevier, vol. 64(2), pages 278-285, January.
    12. Tseng, Lin-Yu & Lin, Ya-Tai, 2009. "A hybrid genetic local search algorithm for the permutation flowshop scheduling problem," European Journal of Operational Research, Elsevier, vol. 198(1), pages 84-92, October.
    13. Vallada, Eva & Ruiz, Rubén, 2010. "Genetic algorithms with path relinking for the minimum tardiness permutation flowshop problem," Omega, Elsevier, vol. 38(1-2), pages 57-67, February.
    14. Bengio, Yoshua & Lodi, Andrea & Prouvost, Antoine, 2021. "Machine learning for combinatorial optimization: A methodological tour d’horizon," European Journal of Operational Research, Elsevier, vol. 290(2), pages 405-421.
    15. S. M. Johnson, 1954. "Optimal two‐ and three‐stage production schedules with setup times included," Naval Research Logistics Quarterly, John Wiley & Sons, vol. 1(1), pages 61-68, March.
    16. Ruiz, Ruben & Stutzle, Thomas, 2007. "A simple and effective iterated greedy algorithm for the permutation flowshop scheduling problem," European Journal of Operational Research, Elsevier, vol. 177(3), pages 2033-2049, March.
    17. Fernandez-Viagas, Victor & Molina-Pariente, Jose M. & Framinan, Jose M., 2020. "Generalised accelerations for insertion-based heuristics in permutation flowshop scheduling," European Journal of Operational Research, Elsevier, vol. 282(3), pages 858-872.
    18. Turkeš, Renata & Sörensen, Kenneth & Hvattum, Lars Magnus, 2021. "Meta-analysis of metaheuristics: Quantifying the effect of adaptiveness in adaptive large neighborhood search," European Journal of Operational Research, Elsevier, vol. 292(2), pages 423-442.
    19. Karimi-Mamaghan, Maryam & Mohammadi, Mehrdad & Jula, Payman & Pirayesh, Amir & Ahmadi, Hadi, 2020. "A learning-based metaheuristic for a multi-objective agile inspection planning model under uncertainty," European Journal of Operational Research, Elsevier, vol. 285(2), pages 513-537.
    20. Nawaz, Muhammad & Enscore Jr, E Emory & Ham, Inyong, 1983. "A heuristic algorithm for the m-machine, n-job flow-shop sequencing problem," Omega, Elsevier, vol. 11(1), pages 91-95.
    21. Benlic, Una & Epitropakis, Michael G. & Burke, Edmund K., 2017. "A hybrid breakout local search and reinforcement learning approach to the vertex separator problem," European Journal of Operational Research, Elsevier, vol. 261(3), pages 803-818.
    22. El-Ghazali Talbi, 2016. "Combining metaheuristics with mathematical programming, constraint programming and machine learning," Annals of Operations Research, Springer, vol. 240(1), pages 171-215, May.
    23. Karimi-Mamaghan, Maryam & Mohammadi, Mehrdad & Meyer, Patrick & Karimi-Mamaghan, Amir Mohammad & Talbi, El-Ghazali, 2022. "Machine learning at the service of meta-heuristics for solving combinatorial optimization problems: A state-of-the-art," European Journal of Operational Research, Elsevier, vol. 296(2), pages 393-422.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ahmad Ebrahimi & Hyun-woo Jeon & Sang-yeop Jung, 2023. "Improving Energy Consumption and Order Tardiness in Picker-to-Part Warehouses with Electric Forklifts: A Comparison of Four Evolutionary Algorithms," Sustainability, MDPI, vol. 15(13), pages 1-28, July.
    2. Lagos, Felipe & Pereira, Jordi, 2024. "Multi-armed bandit-based hyper-heuristics for combinatorial optimization problems," European Journal of Operational Research, Elsevier, vol. 312(1), pages 70-91.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Pagnozzi, Federico & Stützle, Thomas, 2019. "Automatic design of hybrid stochastic local search algorithms for permutation flowshop problems," European Journal of Operational Research, Elsevier, vol. 276(2), pages 409-421.
    2. Karimi-Mamaghan, Maryam & Mohammadi, Mehrdad & Meyer, Patrick & Karimi-Mamaghan, Amir Mohammad & Talbi, El-Ghazali, 2022. "Machine learning at the service of meta-heuristics for solving combinatorial optimization problems: A state-of-the-art," European Journal of Operational Research, Elsevier, vol. 296(2), pages 393-422.
    3. Libralesso, Luc & Focke, Pablo Andres & Secardin, Aurélien & Jost, Vincent, 2022. "Iterative beam search algorithms for the permutation flowshop," European Journal of Operational Research, Elsevier, vol. 301(1), pages 217-234.
    4. Fernandez-Viagas, Victor & Ruiz, Rubén & Framinan, Jose M., 2017. "A new vision of approximate methods for the permutation flowshop to minimise makespan: State-of-the-art and computational evaluation," European Journal of Operational Research, Elsevier, vol. 257(3), pages 707-721.
    5. Fernandez-Viagas, Victor & Talens, Carla & Framinan, Jose M., 2022. "Assembly flowshop scheduling problem: Speed-up procedure and computational evaluation," European Journal of Operational Research, Elsevier, vol. 299(3), pages 869-882.
    6. Lobo, Fernando G. & Bazargani, Mosab & Burke, Edmund K., 2020. "A cutoff time strategy based on the coupon collector’s problem," European Journal of Operational Research, Elsevier, vol. 286(1), pages 101-114.
    7. Brammer, Janis & Lutz, Bernhard & Neumann, Dirk, 2022. "Permutation flow shop scheduling with multiple lines and demand plans using reinforcement learning," European Journal of Operational Research, Elsevier, vol. 299(1), pages 75-86.
    8. Perez-Gonzalez, Paz & Framinan, Jose M., 2024. "A review and classification on distributed permutation flowshop scheduling problems," European Journal of Operational Research, Elsevier, vol. 312(1), pages 1-21.
    9. Benavides, Alexander J. & Vera, Antony, 2022. "The reversibility property in a job-insertion tiebreaker for the permutational flow shop scheduling problem," European Journal of Operational Research, Elsevier, vol. 297(2), pages 407-421.
    10. Ruiz, Rubén & Pan, Quan-Ke & Naderi, Bahman, 2019. "Iterated Greedy methods for the distributed permutation flowshop scheduling problem," Omega, Elsevier, vol. 83(C), pages 213-222.
    11. Gmys, Jan & Mezmaz, Mohand & Melab, Nouredine & Tuyttens, Daniel, 2020. "A computationally efficient Branch-and-Bound algorithm for the permutation flow-shop scheduling problem," European Journal of Operational Research, Elsevier, vol. 284(3), pages 814-833.
    12. Franzin, Alberto & Stützle, Thomas, 2023. "A landscape-based analysis of fixed temperature and simulated annealing," European Journal of Operational Research, Elsevier, vol. 304(2), pages 395-410.
    13. Naderi, Bahman & Ruiz, Rubén, 2014. "A scatter search algorithm for the distributed permutation flowshop scheduling problem," European Journal of Operational Research, Elsevier, vol. 239(2), pages 323-334.
    14. Liu, Weibo & Jin, Yan & Price, Mark, 2017. "A new improved NEH heuristic for permutation flowshop scheduling problems," International Journal of Production Economics, Elsevier, vol. 193(C), pages 21-30.
    15. Pan, Quan-Ke & Ruiz, Rubén, 2012. "Local search methods for the flowshop scheduling problem with flowtime minimization," European Journal of Operational Research, Elsevier, vol. 222(1), pages 31-43.
    16. Li, Xiaoping & Chen, Long & Xu, Haiyan & Gupta, Jatinder N.D., 2015. "Trajectory Scheduling Methods for minimizing total tardiness in a flowshop," Operations Research Perspectives, Elsevier, vol. 2(C), pages 13-23.
    17. Pan, Quan-Ke & Gao, Liang & Li, Xin-Yu & Gao, Kai-Zhou, 2017. "Effective metaheuristics for scheduling a hybrid flowshop with sequence-dependent setup times," Applied Mathematics and Computation, Elsevier, vol. 303(C), pages 89-112.
    18. Sioud, A. & Gagné, C., 2018. "Enhanced migrating birds optimization algorithm for the permutation flow shop problem with sequence dependent setup times," European Journal of Operational Research, Elsevier, vol. 264(1), pages 66-73.
    19. Joaquín Bautista-Valhondo & Rocío Alfaro-Pozo, 2020. "Mixed integer linear programming models for Flow Shop Scheduling with a demand plan of job types," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 28(1), pages 5-23, March.
    20. Kalczynski, Pawel J. & Kamburowski, Jerzy, 2009. "An empirical analysis of the optimality rate of flow shop heuristics," European Journal of Operational Research, Elsevier, vol. 198(1), pages 93-101, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:304:y:2023:i:3:p:1296-1330. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.