Markov decision process model for patient admission decision at an emergency department under a surge demand

We study an admission control problem for patients arriving at an emergency department in the aftermath of a mass casualty incident. A finite horizon Markov decision process (MDP) model is formulated to determine patient admission decisions. In particular, our model considers the time-dependent arrival of patients and time-dependent reward function. We also consider a policy restriction that immediate-patients should be admitted as long as there is available beds. The MDP model has a continuous state space, and we solve the model by using a state discretization technique and obtain numerical solutions. Structural properties of an optimal policy are reviewed, and the structures observed in the numerical solutions are explained accordingly. Experimental results with virtual patient arrival scenarios demonstrates the performance and advantage of optimal policies obtained from the MDP model.