Development and internal validation of prediction models for future hospital care utilization by patients with multimorbidity using electronic health record data

Objective: To develop and internally validate prediction models for future hospital care utilization in patients with multiple chronic conditions. Design: Retrospective cohort study. Setting: A teaching hospital in the Netherlands (542 beds) Participants: All adult patients (n = 18.180) who received care at the outpatient clinic in 2017 for two chronic diagnoses or more (including oncological diagnoses) and who returned for hospital care or outpatient clinical care in 2018. Development and validation using a stratified random split-sample (n = 12.120 for development, n = 6.060 for internal validation). Outcomes: ≥2 emergency department visits in 2018, ≥1 hospitalization in 2018 and ≥12 outpatient visits in 2018. Statistical analysis: Multivariable logistic regression with forward selection. Results: Evaluation of the models’ performance showed c-statistics of 0.70 (95% CI 0.69–0.72) for the hospitalization model, 0.72 (95% CI 0.70–0.74) for the ED visits model and 0.76 (95% 0.74–0.77) for the outpatient visits model. With regard to calibration, there was agreement between lower predicted and observed probability for all models, but the models overestimated the probability for patients with higher predicted probabilities. Conclusions: These models showed promising results for further development of prediction models for future healthcare utilization using data from local electronic health records. This could be the first step in developing automated alert systems in electronic health records for identifying patients with multimorbidity with higher risk for high healthcare utilization, who might benefit from a more integrated care approach.