Koopman-enhanced offline reinforcement learning for robust multi-echelon inventory control under demand distribution shift

PurposeMulti-echelon inventory optimization (MEIO) under demand distribution shift remains an open challenge in supply chain management. Classical rule-based policies are effective when calibrated online but cannot exploit historical data structure. This paper investigates whether a Koopman operator world model, applied as a structured latent dynamics model within an offline reinforcement learning (RL) framework, can improve the out-of-distribution (OOD) robustness of learned inventory policies and reduce temporal difference (TD)-target variance during training.Design/methodology/approachWe propose Koopman-enhanced Multi-Echelon Inventory Optimization (K-MEIO), a two-stage framework combining a Deep Koopman World Model (DKWM) with Implicit Q-Learning (IQL). The DKWM learns a finite-dimensional linear embedding of the multi-echelon inventory dynamics, enabling analytically tractable TD targets with 12.4% lower variance than direct next-state encoding. The Koopman-enhanced Implicit Q-Learning policy is trained entirely on simulated offline transitions calibrated from University of California, Irvine Online Retail and Rossmann Store Sales demand statistics. Ablation experiments across five expectile values, three training seeds, and three OOD stress scenarios are reported, together with full cost breakdowns.FindingsUnder standard evaluation, K-MEIO achieves service level 0.995 with zero capacity violations. Across three training seeds, K-MEIO achieves OOD robustness ratio 0.861 ± 0.039 under a 3× promotional demand spike and 0.882 ± 0.058 under a permanent demand mean shift, compared to 0.762 ± 0.153 and 0.842 ± 0.093 for vanilla IQL. K-MEIO shows consistently lower seed-to-seed variance, suggesting more stable training dynamics. A historically calibrated base-stock policy (Hist-BSP) outperforms all offline methods in both standard and stress evaluations. Cost breakdown confirms that K-MEIO's degradation under stress is driven entirely by stockout cost, not excess inventory.Research limitations/implicationsThe offline dataset consists of simulated transitions calibrated using real retail demand statistics, not actual historical inventory records. The Koopman spectral radius constraint is a regularization device and does not constitute a formal stability guarantee for the true inventory system. The 12.4% variance reduction produces modest but consistent OOD robustness gains across seeds.Practical implicationsIn supply chain contexts where promotional demand spikes or demand mean shifts are common and online policy recalibration is impractical, K-MEIO offers more stable policy behavior than vanilla IQL (seed std. ±0.039 vs. ± 0.153 in S1). A historically calibrated BSP remains a competitive offline benchmark and should be the primary comparison point.Originality/valueThis paper provides the first systematic evaluation of Koopman-structured TD targets for offline inventory RL under demand distribution shift, with direct measurement of TD-target variance reduction, a multi-seed robustness analysis, and a fair historically calibrated BSP baseline. The corrected mechanistic explanation clarifies that the Koopman advantage operates at training time through variance reduction, not at evaluation time through OOD encoding.