Accelerated Scheduling Heuristics for Reinforcement Learning Approaches Applied to the Westenberger–Kallrath Problem

In the chemical process industry (CPI), production planning and scheduling (PPS) problems show a variety of characteristics which increases the complexity of corresponding modeling approaches and solution techniques. For recent years, the concept of reinforcement learning (RL) has been evaluated for its effectiveness to solve PPS problems for multiple case studies in discrete manufacturing. Our study delves into the utilization of RL to address the Westenberger–Kallrath (WK) problem which is a prominent benchmark for chemical production planning. The proposed solution approach decouples the lotsizing part from the scheduling activity via a custom MRP heuristic which creates chains of batches linked to customer orders. We train an RL agent to sequence those chains with the objective to minimize overall makespan. Building on our previous work, which used a custom forward scheduling heuristic with an internal inventory projection, we propose a new scheduling logic based on minimum waiting times between the multi-stage processes and their assigned resources. We provide a CPU time comparison between two implementation approaches, which proves the efficiency of our new method. Using the new logic in RL agent training, we detect parameter configuration as key algorithmic challenge for further studies.