Controlling the Continuos Positive Airway Pressure-Device Using Partial Observable Markov Decision Processes

Summary Partial Observable Markov Decision Processes (POMDP's) allow finding an optimal control of a hidden Markov model. We develop within this framework a control of a CPAP-device (Continuous Positive Airway Pressure). People who suffer from an obstructive sleep apnoea syndrome may use such a device by which a small additive air pressure is applied to the patient through a mask so that the occlusion of the respiratory tract is prevented. Different values of the pressure will correspond to different actions in the POMDP-approach. We study the performance of a control determined by a POMDP-approach in comparison with other, more traditional control strategies. In commercial devices the applied pressure is held on a constant level, carefully determined in a sleep laboratory. More recently, the pressure is controlled by a classification of the airflow during breathing and then choosing the action in dependence on the classification features. Within the POMDP approach, we introduce the unobservable state of the airway as the hidden state and regard the features of the airflow during breathing as the observations. The parameters of the hidden models are chosen with help of experience from laboratories, where such devices are developed. We will compare the POMDP control with direct control strategies in dependence of parameters of a POMDP model. We will show that the advantage of POMDP control is substantial in general, however, that there are also cases, where the performance of the belief control does not differ too much from some direct control. We discuss, when this does happen and thus find a criterion for the implementation of the POMDP-control instead of a direct control.