Explainable detection of adverse drug reaction with imbalanced data distribution

Analysis of health-related texts can be used to detect adverse drug reactions (ADR). The greatest challenge for ADR detection lies in imbalanced data distributions where words related to ADR symptoms are often minority classes. As a result, trained models tend to converge to a point that strongly biases towards the majority class and then ignores the minority class. Since the most used cross-entropy criteria is an approximation to accuracy, the model focuses more readily on the majority class to achieve high accuracy. To address this issue, existing methods apply either oversampling or down-sampling strategies to balance the data distribution and exploit the most difficult samples of the minority class. However, increasing or reducing the number of individual tokens alone in sequence labeling tasks will result in the loss of the syntactic relations of the sentence. This paper proposes a weighted variant of conditional random field (CRF) for data-imbalanced sequence labeling tasks. Such a weighting strategy can alleviate data distribution imbalances between majority and minority classes. Instead of using softmax in the output layer, the CRF can capture the relationship of labels between tokens. The locally interpretable model-agnostic explanations (LIME) algorithm was applied to investigate performance differences between models with and without the weighted loss function. Experimental results on two different ADR tasks show that the proposed model outperforms previously proposed sequence labeling methods.Author summary: Post-marketing drug safety surveillance offers the chance to detect serious ADRs resulting in hospitalization and ADRs occurring in patients, e.g., patients with high comorbidity and receiving drugs that are administered only in hospitals. This monitoring has traditionally been accomplished by surveying users. Recently, the automatically recording ADR of users in social media can greatly help biopharmaceutical enterprises to improve their products. Previous methods of name entity recognition in natural language processing were usually performed on the corpora with a balanced data distribution. Conversely, the datasets for ADR detection are extremely imbalanced. As a result, the detector tends to ignore the ADR symptoms and the related indications, which are more important. In this study, we propose a weighted CRF model based on BERT for the detection task of ADR. A weighted variant of the Viterbi Algorithm is implemented to assign more weight to the minority class, forcing the model to pay more attention to minority classes to ensure effective detection. The results suggested that the proposed method provides a significant performance boost without changing the model architecture in imbalanced-data tasks.