Towards Automating Adverse Event Review: A Prediction Model for Case Report Utility

Introduction The rapidly expanding size of the Food and Drug Administration’s (FDA) Adverse Event Reporting System database requires modernized pharmacovigilance practices. Techniques to systematically identify high utility individual case safety reports (ICSRs) will support safety signal management. Objectives The aim of this study was to develop and validate a model predictive of an ICSR’s pharmacovigilance utility (PVU). Methods PVU was operationalized as an ICSR’s inclusion in an FDA-authored pharmacovigilance review’s case series supporting a recommendation to modify product labeling. Multivariable logistic regression models were used to examine the association between PVU and ICSR features. The best performing model was selected for bootstrapping validation. As a sensitivity analysis, we evaluated the model’s performance across subgroups of safety issues. Results We identified 10,381 ICSRs evaluated in 69 pharmacovigilance reviews, of which 2115 ICSRs were included in a case series. The strongest predictors of ICSR inclusion were reporting of a designated medical event (odds ratio (OR) 1.93, 95% CI 1.54–2.43) and positive dechallenge (OR 1.67, 95% CI 1.50–1.87). The strongest predictors of ICSR exclusion were death reported as the only outcome (OR 2.72, 95% CI 1.76–4.35), more than three suspect products (OR 2.69, 95% CI 2.23–3.24), and > 15 preferred terms reported (OR 2.69, 95% CI 1.90–3.82). The validated model showed modest discriminative ability (C-statistic of 0.71). Our sensitivity analysis demonstrated heterogeneity in model performance by safety issue (C-statistic range 0.58–0.74). Conclusions Our model demonstrated the feasibility of developing a tool predictive of ICSR utility. The model’s modest discriminative ability highlights opportunities for further enhancement and suggests algorithms tailored to safety issues may be beneficial.