AI-Assisted Management Triage in Urological Emergencies: A Prospective Comparison With an Expert Physician Panel

Artificial intelligence (AI)-based decision-support tools are increasingly explored for emergency triage, but their performance in acute urological care remains insufficiently characterized. Emergency urology includes both systemic high-acuity presentations and time-critical anatomical emergencies, requiring accurate stratification of management intensity. This prospective observational study evaluated the agreement between a Microsoft Copilot Studio-based AI decision-support agent and an expert physician panel for management-intensity triage in urological emergencies. We included 100 consecutive patients presenting to the urological emergency unit of a Romanian tertiary hospital. Each case was independently assessed by the AI agent and by a five-member physician panel. Recommendations were classified into four predefined management-intensity categories: outpatient/conservative management, hospital-based treatment/imaging/monitoring, urgent urological intervention/source control, and ICU/sepsis-level emergency management. Agreement was assessed using percentage agreement, Cohen’s kappa, weighted kappa, and directional mismatch analysis. The cohort included 100 patients across 13 diagnostic categories; 76.0% were male, and the mean age was 49.2 ± 18.0 years. Overall AI–panel agreement was 66.0% (95% CI: 56.3–74.5%). Cohen’s kappa was 0.510, indicating moderate agreement, while linear and quadratic weighted kappa values were 0.585 and 0.672, respectively. Discordance occurred in 34 cases and was mainly attributable to AI under-escalation. The AI correctly classified all ICU/sepsis-level cases but showed low sensitivity for urgent urological intervention/source control (30.4%), particularly in testicular torsion. The AI agent showed moderate agreement with expert-panel decisions and appeared reliable for sepsis-level escalation, but clinically relevant under-escalation occurred in time-critical anatomical emergencies. Physician oversight, diagnosis-specific safeguards, and external validation remain essential before clinical implementation.