Deep learning for blood glucose prediction: Reproducibility challenges and factors affecting differential performance

Blood glucose prediction is a critical component of next-generation diabetes technologies, such as artificial pancreas systems, where reliable performance is essential for safety and effectiveness. Although deep learning methods have achieved promising advances in this area, a critical gap remains in understanding the reproducibility and generalizability of these methods. To contextualize the gap, this study reviewed 67 recent papers that proposed a deep learning method for glucose prediction to identify key reproducibility challenges. Next, we adopted a standardized framework, encompassing technical, statistical, and conceptual reproducibility evaluations, to experimentally assess the reproducibility of eight representative deep learning methods. To achieve this, we reimplemented and evaluated these eight deep learning methods using over 1.36 million continuous glucose monitoring samples (5,061 days) from 128 individuals with type 1 diabetes across three public datasets: OhioT1DM, DiaTrend, and T1DEXI. We found that even though these models demonstrated good technical and statistical reproducibility, their conceptual reproducibility—the ability to generalize to datasets with different diabetes management patterns—was limited. Further analyses revealed that each model’s overall prediction performance was strongly influenced by individual glycemic control, with higher prediction errors observed among participants with lower time with blood glucose in the target range (70–180 mg/dL). This study identified key reproducibility challenges associated with current blood glucose prediction methods within type 1 diabetes populations, highlighting the need for increased transparency, dataset diversity, standardized evaluation practices, and code accessibility to ensure reproducible and reliable models for blood glucose prediction.Author summary: Managing type 1 diabetes requires individuals to frequently monitor their blood glucose levels and constantly consider how decisions related to food, activity, insulin use, and more affect their blood glucose levels. Accurate blood glucose prediction is critical to inform diabetes management strategies and mitigate adverse events. Consequently, many researchers have explored deep learning models for predicting future blood glucose levels, with the goal of providing earlier warnings of adverse events and personalized guidance for diabetes management. However, despite the progress, little effort exists on evaluating the reproducibility of published models for blood glucose prediction. To address this gap, we evaluated the reproducibility of eight state-of-the-art deep learning models using three publicly available datasets from distinct populations with type 1 diabetes. This evaluation allowed us to examine not only whether the reported results could be accurately replicated but also whether these models could generalize to new groups of individuals with different blood glucose management strategies and outcomes. Findings from this study revealed several barriers that limit model reproducibility. Furthermore, we found that the performance of many models was significantly affected by the heterogeneity of glucose management across individuals. Our work highlights the importance of transparent research practices and evaluation across diverse datasets to ensure that future glucose prediction models are reliable and equitable for all people living with diabetes.