A multi-objective reinforcement learning approach for resequencing scheduling problems in automotive manufacturing systems

This study investigated a multi-objective resequencing scheduling problem in the automotive manufacturing systems due to operational requirements on the color-batching of the paint shop and sequential requirements on the sequence adherence of an assembly shop. Resequencing cars as color-oriented batches reduced the costs of color changes and operational costs for paint shops. Also, assembly shops required paint shops to complete cars with fewer delays so that high sequence adherence with its demand was assured. Based on real-world applications, we investigated two contradictory objectives-color change costs and sequence tardiness-in a single-machine flowshop scheduling environment. A multi-objective-deep-Q-network algorithm was developed to determine the Pareto frontier. Reward shaping was designed to improve the convergence of the neural network. The 2D-folded-normal distribution was designed to sample the preference, which made the exploration and exploitation of the neural network more comprehensive and improved the training efficiency. Two experiments were conducted and showed that the proposed approach outperformed the meta-heuristic algorithm and the envelope Q-learning algorithm in solving time, performance, the convergence of the neural network, and the diversity of the Pareto frontier. Therefore, the proposed approach can be used in automotive paint shops to improve scheduling efficiency and reduce operational costs.