CPLLM: Clinical prediction with large language models

We present Clinical Prediction with Large Language Models (CPLLM), a method that involves fine-tuning a pre-trained Large Language Model (LLM) for predicting clinical disease and readmission. We utilized quantization and fine-tuned the LLM using prompts. For diagnostic predictions, we predicted whether patients would be diagnosed with a target disease during their next visit or in the subsequent diagnosis, leveraging their historical medical records. We compared our results to various baselines, including Retain and Med-BERT, the latter of which is the current state-of-the-art model for disease prediction using temporal structured EHR data. In addition, we also evaluated CPLLM’s utility in predicting hospital readmission and compared our method’s performance with benchmark baselines. Our experiments ultimately revealed that our proposed method, CPLLM, surpasses all the tested models in terms of PR-AUC and ROC-AUC metrics, providing state-of-the-art performance as a tool for predicting disease diagnosis and patient hospital readmission without requiring pre-training on medical data. Such a method can be easily implemented and integrated into the clinical workflow to help care providers plan next steps for their patients.Author summary: We introduce Clinical Prediction with Large Language Models (CPLLM), a novel method that fine-tunes a pre-trained Large Language Model (LLM) to enhance predictions of clinical diseases and patient readmissions. By leveraging historical medical records, we aimed to predict whether patients will be diagnosed with a specific disease or be readmitted. Our method is compared against the current state-of-the-art model for using structured electronic health record (EHR) data. Our findings demonstrate that CPLLM significantly outperforms state-of-the-art models in both PR-AUC and ROC-AUC metrics. Additionally, our method does not require pre-training on clinical data, making it straightforward to implement with existing LLMs. By integrating CPLLM, healthcare providers can make informed decisions about patient care, ultimately leading to better outcomes. CPLLM can be readily adopted within clinical workflows, assisting care providers in planning appropriate next steps for their patients.