Automated detection and prediction of suicidal behavior from clinical notes using deep learning

Background: Deep learning approaches have tremendous potential to improve the predictive power of traditional suicide prediction models to detect and predict intentional self-harm (ISH). Existing research is limited by a general lack of consistent performance and replicability across sites. We aimed to validate a deep learning approach used in previous research to detect and predict ISH using clinical note text and evaluate its generalizability to other academic medical centers. Methods: We extracted clinical notes from electronic health records (EHRs) of 1,538 patients with International Classification of Diseases codes for ISH and 3,012 matched controls without ISH codes. We evaluated the performance of two traditional bag-of-words models (i.e., Naïve Bayes, Random Forest) and two convolutional neural network (CNN) models including randomly initialized (CNNr) and pre-trained Word2Vec initialized (CNNw) weights to detect ISH within 24 hours of and predict ISH from clinical notes 1–6 months before the first ISH event. Results: In detecting concurrent ISH, both CNN models outperformed bag-of-words models with AUCs of.99 and F1 scores of 0.94. In predicting future ISH, the CNN models outperformed Naïve Bayes models with AUCs of 0.81–0.82 and F1 scores of 0.61−.64. Conclusions: We demonstrated that leveraging EHRs with a well-defined set of ISH ICD codes to train deep learning models to detect and predict ISH using clinical note text is feasible and replicable at more than one institution. Future work will examine this approach across multiple sites under less controlled settings using both structured and unstructured EHR data.