Patient-Type Bayes-Adaptive Treatment Plans

Patient heterogeneity in disease progression is prevalent in many settings. Treatment decisions that explicitly consider this heterogeneity can lower the cost of care and improve outcomes by providing the right care for the right patient at the right time. In this paper, we analyze the problem of designing ongoing treatment plans for a population with heterogeneity in disease progression and response to medical interventions. We create a model that learns the patient type by monitoring the patient health over time and updates a patient’s treatment plan according to the gathered information. We formulate the problem as a multivariate state-space partially observable Markov decision process (POMDP) and provide structural properties of the value function, as well as the optimal policy. We extend this modeling framework to a general class of treatment initiation problems where there is a stochastic lead time before a treatment becomes available or effective. As a case study, we develop a data-driven decision-analytic model to study the optimal timing of vascular access surgery for patients with progressive chronic kidney disease, and we establish policies that consider a patient’s rate of disease progression in addition to the kidney health state. To circumvent the curse of dimensionality of the POMDP, we develop several approximate policies, as well as simpler heuristics, and evaluate them against a high-quality lower bound. Through a numerical study and several sensitivity analyses, we establish the high quality and robustness of an approximate policy that we develop. We provide further policy insights that sharpen existing guidelines for the case-study problem.