Multi-Armed Bandits with Endogenous Learning Curves: An Application to Split Liver Transplantation

Problem Definition: Proficiency in many sophisticated tasks is attained through experience-based learning, in other words, learning by doing. For example, transplant centers’ surgical teams need to practice difficult surgeries to master the skills required. Meanwhile, this experience-based learning may affect other stakeholders, such as patients eligible for transplant surgeries, and require resources, including scarce organs and continual efforts. To ensure that patients have excellent outcomes and equitable access to organs, the organ allocation authority needs to quickly identify and develop medical teams with high aptitudes. This entails striking a balance between exploring surgical combinations with initially unknown full potential and exploiting existing knowledge based on observed outcomes. Methodology/results: We formulate a multi-armed bandit (MAB) model in which parametric learning curves are embedded in the reward functions to capture endogenous experience-based learning. In addition, our model includes provisions ensuring that the choices of arms are subject to fairness constraints to guarantee equity. To solve our MAB problem, we propose the L-UCB and FL-UCB algorithms, variants of the upper confidence bound (UCB) algorithm that attain the optimal O ( log t ) regret on problems enhanced with experience-based learning and fairness concerns. We demonstrate our model and algorithms on the split liver transplantation (SLT) allocation problem, showing that our algorithms have superior numerical performance compared with standard bandit algorithms in a setting where experience-based learning and fairness concerns exist. Managerial implications: From a methodological point of view, our proposed MAB model and algorithms are generic and have broad application prospects. From an application standpoint, our algorithms could be applied to help evaluate potential strategies to increase the proliferation of SLT and other technically difficult procedures.