Hierarchical production control and distribution planning under retail uncertainty with reinforcement learning

Effective coordination between production control and distribution planning is critical in supply chain management. However, existing research mainly focuses on responding to stochastic demand, while the impact of uncertain retail capabilities is often overlooked. This study proposes a hierarchical framework that integrates and coordinates production control and distribution planning while explicitly addressing the uncertainty of retail capabilities. Specifically, we develop a reinforcement learning (RL) algorithm that learns stochastic retail capabilities under adaptive production control (upper level) and distribution planning (lower level). This retail information is then fed into the hierarchical control framework, which enhances the performance of both control layers to maximise system profit while considering opportunity costs and holding costs. Moreover, we incorporate a novel holding function based on the exponential penalty term into the reward function to effectively enforce the side constraint of inventory capacity. This approach enables the RL algorithm to derive feasible production policies and thereby enhance the training process. We evaluate the proposed hierarchical controller through a case study utilising real-world transaction data from the steel manufacturing industry. The results demonstrate that the accurate identification of retail capabilities can facilitate inventory management under stochastic market conditions. Furthermore, the hierarchical framework can improve overall profits by coordinating production control actions under different retail strategies.