Patient Scheduling with Approximate Dynamic Programming for Optimization of Health Care Services

Model that prescribes the optimal appointment date for a patient at the moment this patient makes his request at the outpatient clinic is developed. We categorized patients into two. The first category is concerned with patients with a maximum recommended waiting time. For these types of patient, the sooner these patients are scheduled the better and when the maximum recommended waiting time is exceeded, extra costs are incurred. The other category is characterized by a specific appointment time. The closer the scheduled appointment time is to the specific appointment time, the lower the costs. The objective is to minimize the long-run expected average cost. We modelled the scheduling process as a Markov Decision Process (MDP). we then apply the Bellman Error Minimization (BEM) method as an Approximate Dynamic Programming technique in order to derive an estimate of the optimal value function of our MDP of which the optimal policy (appointment date) can be derived. To determine the set of representative states, which is an element of the BEM method, we use the k-means algorithm. We test several approximation functions and find an approximation function that outperforms all other functions in the scheduling process over four, six, and eight working days. The Approximation Function B gives the near optimal appointment date for patients when appointments are requested. In general, it holds that the higher the arrival rate of patients at the outpatient clinic, the better our BEM method performs. But if the arrival rate reaches a certain value the load of the system becomes that high that it does not matter what policy is applied, since many patients have to be rejected.