Flexible material handling system for multi-load autonomous mobile robots in manufacturing environments: a hierarchical reinforcement learning approach

The increasing complexity of customer demands has led to the implementation of flexible job-shop scheduling and automated material handling systems across manufacturing sectors. In particular, advances in robotic technology have made autonomous mobile robots (AMRs) essential for material handling tasks within these sectors. The capability of free movement, path planning, and loading multiple work-in-processes (WIPs) can significantly enhance the efficiency of material handling operations. However, the full flexibility of AMRs cannot be utilised when their decisions regarding the sequence of loading and unloading multiple WIPs are made by specific rule-based operations, resulting in inefficiencies in the throughput of WIPs in manufacturing environments. To address this inefficiency, we introduce a hierarchical reinforcement learning algorithm to optimise material handling with AMRs, thereby maximising the throughput of WIPs. In this approach, a graph attention network (GAT) serves as an encoder for the hierarchical reinforcement learning (HRL) input, effectively capturing the complex relationships between different nodes. Computational experiments demonstrate that our approach enhances the efficiency of the material handling system more effectively than existing rule-based methods.