Learning-based hybrid algorithms for container relocation problem with storage plan

Container relocation poses a challenge in terminals, underscoring the importance of effective strategies to maintain efficient cargo flow. This paper tackles the Container Relocation Problem with Storage Plan (CRPSP) by integrating exact algorithms and reinforcement learning. We formulate the problem with a mixed-integer programming model and employ an A* algorithm to generate optimal solutions. We then design a reinforcement learning PPO model with an attention mechanism. A key contribution is the designed reward mechanism based on a lower-bound evaluation function of the A* algorithm, which significantly accelerates the convergence of the reinforcement learning model. Additionally, ensemble learning techniques, specifically Stacking, are first used to integrate multiple reinforcement learning models based on optimal solutions. Numerical experiments demonstrate notable improvements in convergence speed and robustness, highlighting the potential of combining operations research methods and machine learning for complex NP-hard problems.