Sequential resource allocation for humanitarian operations using approximate dynamic programming

In humanitarian and nonprofit operations, distributing aid such as food, shelter, and medical supplies becomes challenging in an online setting, where the future demand is unknown, since allocation decisions are made in real time as uncertainties unfold. Being overly conservative in allocating items at the beginning of the supply chain to save stock for fulfilling demand further down the supply chain increases the likelihood of unallocated items (waste). On the other hand, fully addressing the demand of nodes in the earlier stages of the supply chain may negatively impact the equity of the allocation policy, as downstream nodes may receive significantly fewer items in proportion to their demand. This study proposes a framework for modeling the sequential decisions involved in this online resource allocation problem as a Markov Decision Process (MDP). Given that the size of the state–action space can become very large for this problem, standard dynamic programming methods in the reinforcement learning domain reach their limits, so using Approximate Dynamic Programming (ADP) is a practical solution. In this study, two methods of measuring downstream uncertainty are proposed, and Policy Function Approximation (PFA) is used to develop an optimal allocation policy. Numerical results and the application of the proposed model to the Food Bank of Southern Tier in New York suggest a reasonable balance between maximizing efficiency (minimizing the waste of unallocated items) and ensuring an equitable allocation.