Towards artificial intelligence-based disease prediction algorithms that comprehensively leverage and continuously learn from real-world clinical tabular data systems

This manuscript presents a proof-of-concept for a generalizable strategy, the full algorithm, designed to estimate disease risk using real-world clinical tabular data systems, such as electronic health records (EHR) or claims databases. By integrating classic statistical methods and modern artificial intelligence techniques, this strategy automates the production of a disease prediction model that comprehensively reflects the dynamics contained within the underlying data system. Specifically, the full algorithm parses through every facet of the data (e.g., encounters, diagnoses, procedures, medications, labs, chief complaints, flowsheets, vital signs, demographics, etc.), selects which factors to retain as predictor variables by evaluating the data empirically against statistical criteria, structures and formats the retained data into time-series, trains a neural network-based prediction model, then subsequently applies this model to current patients to generate risk estimates. A distinguishing feature of the proposed strategy is that it produces a self-adaptive prediction system, capable of evolving the prediction mechanism in response to changes within the data: as newly collected data expand/modify the dataset organically, the prediction mechanism automatically evolves to reflect these changes. Moreover, the full algorithm operates without the need for a-priori data curation and aims to harness all informative risk and protective factors within the real-world data. This stands in contrast to traditional approaches, which often rely on highly curated datasets and domain expertise to build static prediction models based solely on well-known risk factors. As a proof-of-concept, we codified the full algorithm and tasked it with estimating 12-month risk of initial stroke or myocardial infarction using our hospital’s real-world EHR. A 66-month pseudo-prospective validation was conducted using records from 558,105 patients spanning April 2015 to September 2023, totalling 3,424,060 patient-months. Area under the receiver operating characteristic curve (AUROC) values ranged from .830 to .909, with an improving trend over time. Odds ratios describing model precision for patients 1–100 and 101–200 (when ranked by estimated risk) ranged from 15.3 to 48.1 and 7.2 to 45.0, respectively, with both groups showing improving trends over time. Findings suggest the feasibility of developing high-performing disease risk calculators in the proposed manner.Author summary: The increasing prominence of disease risk calculators for preventive health is hindered by generalizability issues and narrow reliance on traditionally considered risk factors. We present a generalizable strategy/algorithm that automates the production of risk calculators that comprehensively leverage and continuously learn from real-world local tabular patient data systems, thereby ensuring that the resulting predictions reflect the contemporary health dynamics relevant to the local populations. Our local-focused approach contrasts with strategies that use increasingly extensive but less locally relevant data during model development. By applying our algorithm to a single hospital’s electronic health record system, we demonstrate the feasibility of our approach, as well as the potential for it to produce high-performing risk estimation systems in a real-world setting. Though we present a particular specification, our approach is general and should be adapted to reflect the specifics of the local context and endpoint of interest. Additionally, because our strategy aims to utilize all available informative tabular data when calculating risk, as local tabular data systems evolve to include new sources of data (e.g., omics, health monitor, structured data extracted autonomously from physician notes or imaging studies), our strategy can be readily expanded to include these new data, potentially enhancing risk estimates.