A penalized maximum likelihood approach to deal with latent state separation in hidden Markov models with covariates and lagged responses

A penalized maximum likelihood estimation approach is proposed for discrete-time hidden Markov models in which the manifest distribution depends on covariates and the lagged response. The proposed method addresses the issue of latent state separation, typically arising with binary responses or categorical response variables with a limited number of categories, which leads to extremely large estimates of the support points of the latent variable distribution. We also propose a cross-validation approach for jointly selecting the number of latent states of the model and the strength of the likelihood penalization. The approach is validated through a deep simulation study aimed at comparing parameter estimation accuracy and computational efficiency across different estimation procedures. Finally, we illustrate the proposed approach by analyzing longitudinal data collected during spinal anesthesia, including covariates, with the aim of monitoring the occurrence of hypotension in certain patients.