Multi-agent deep reinforcement learning for ordering and inventory allocation in a decentralized two-echelon dual-channel supply chain

The surge of dual-channel distribution intensifies inventory competition and channel conflict between manufacturers and independent retailers, rendering traditional inventory allocation mechanisms inadequate in decentralized scenarios. To address this, we explore a dynamic ordering and inventory allocation problem within a decentralized dual-channel supply chain characterized by capacity constraints and positive lead times, where neither the demand distribution nor the order quantities from downstream entities are known. We formulate the interaction between the manufacturer and the retailer as a non-cooperative partially observable stochastic game and propose a novel hybrid-action-space and heterogeneous-agent deep deterministic policy gradient (HA2DDPG) algorithm. Unlike classical deterministic policy gradient methods that are restricted to continuous action spaces, we introduce a discrete gradient estimator, called Gumbel-Softmax reparameterization, to handle hybrid action spaces. Experiments on one synthetic dataset and two real-world datasets demonstrate that HA2DDPG outperforms two state-of-the-art multi-agent deep reinforcement learning algorithms, two base-stock heuristics, two Large Language Models (LLMs), and a practical human-driven policy in most cases. Relative to an oracle with perfect demand foresight, HA2DDPG incurs an average profit loss of only 4.7% for the manufacturer and 9.4% for the retailer. Moreover, our findings suggest that LLMs may perpetuate initially inaccurate output structures under marginal input perturbations. We also develop an ensemble and explainable learning framework to enhance policy comprehensibility. By leveraging univariate decision trees and ordinary least squares regression, the framework translates state-action pairs learned by HA2DDPG into portable and explainable decision rules.