Implementation of an opioid use disorder (OUD) machine-learning phenotype in real-time for the ADAPT clinical trial

We developed and deploy a real‑time, electronic health record‑integrated machine learning phenotype to identify emergency department patients with opioid use disorder for prospective clinical trial screening and buprenorphine initiation. We conducted a multi‑phase study across three emergency departments in a single United States health system from 2014 to 2025. Using visit‑level data available at or before triage, we trained a random‑forest classifier to estimate opioid use disorder risk and embedded scoring in the electronic health record to trigger point‑of‑care alerts for trial eligibility review. A high-specificity computable training label supported retrospective development, and a clinician gold-standard reference was established through structured chart review. Performance was summarized using discrimination, calibration, and threshold-based classification metrics; prospective validation used a stratified random sample of flagged and unflagged encounters. Retrospective discrimination compared to the high-specificity computable training label was high (area under the receiver operating characteristic curve, 0.99; 95% CI, 0.98–0.99; area under the precision-recall curve, 0.92; 95% CI, 0.90–0.94), and calibration plots informed operating‑point selection for real‑time use. In prospective gold‑standard validation (n = 217), 89.3% of model-positive encounters and 95.7% of model-negative encounters agreed with physician adjudication at the prespecified threshold. Because the sample was stratified on model prediction, these are within-stratum agreement estimates; weighting them by the source-population flag rate (28,284 of 866,569 eligible encounters, 3.26%) yields design-weighted estimates of population-level prevalence (7.1%), sensitivity (0.40, 95% CI 0.24-0.64), and specificity (0.996). An electronic health record‑embedded, machine learning phenotype can accurately and feasibly identify emergency department patients with opioid use disorder in real time, streamlining clinical trial enrollment and treatment initiation. Ongoing work will report operational metrics (e.g., alert volume and latency), monitor performance drift and equity across subgroups, and evaluate downstream clinical and trial outcomes.Author summary: In this study, we wanted to find a better way to identify people visiting the emergency department who may be living with opioid use problems. These patients are often hard to recognize quickly, even though timely support, including offering effective medications, can greatly improve their health and safety. Traditional approaches rely heavily on clinicians noticing certain patterns in real time, which can be challenging during busy emergency care. To address this gap, we examined over a decade of information from a large health system and developed a clinical decision support tool that reviews routine information collected at the start of an emergency visit. The tool gives clinicians a simple, real-time signal when a patient may benefit from additional evaluation or treatment. Our early experience shows that this approach can help teams identify patients more consistently and connect them to care more quickly. We hope this work will make it easier for hospitals to support people affected by opioid use and to run clinical studies that improve future treatment options.