On the integration of reinforcement learning and simulated annealing for the parallel batch scheduling problem with setups

Motivated by semiconductor applications, where wafer lots are grouped into families and processed on batch machines, this paper addresses a generalized unrelated parallel-batch scheduling problem. The goal is to minimize total completion time (flow time) while considering family- and machine-dependent setup times. We propose a mixed-integer programming formulation, establish a necessary condition for optimal schedules, and develop a polynomial-time heuristic for batching and sequencing. We also evaluate Q-Learning, a model-free reinforcement learning algorithm, for neighborhood selection within two Simulated Annealing-based metaheuristics: Stochastic Local Search (SLS) and Adaptive Large Neighborhood Search (ALNS). Results show that SLS and ALNS achieve better solutions and faster convergence compared to existing approaches. Finally, we conclude that while Q-Learning has the potential to improve solution quality in certain cases, it also increases the complexity of the algorithms, making them harder to configure and scale.