Inventory-constrained online learning for revenue management with delayed feedback

Delayed feedback is a prevalent challenge in modern logistics and transportation systems, especially on digital retail platforms. This paper investigates an online learning and pricing problem characterized by aggregated and anonymous delays. In this setting, neither demand nor revenue is immediately observable following a pricing decision; instead, these metrics become available to the retailer only after some stochastic delay. The retailer also faces an initial inventory constraint, creating a complex exploration-exploitation trade-off among learning demand, generating revenue, and managing inventory. To address this challenge, we propose a novel batch-based learning algorithm, referred to as Bandits with Dual Mirror Descent (BUD for short), which integrates mirror descent with bandit control. The algorithm employs a carefully designed batch structure to isolate the impact of delayed feedback, while combining Upper Confidence Bound (UCB) for pricing with dual updates for inventory management. Our theoretical analysis shows that the regret (defined as the revenue gap between the optimal policy and the learning algorithm) of BUD grows sublinearly with the selling horizon and matches the known lower bounds in both bandit with delays and online pricing problems. We conducted numerical experiments to demonstrate that the regret of BUD converges to 0 in various scenarios.