Development, internal and external evaluation of an artificial intelligence algorithm for child growth monitoring in primary care

Our goal was to improve growth monitoring in children by developing and evaluating an artificial intelligence (AI) algorithm that can detect abnormal growth. We used pre-diagnosis height measurements for children with a diagnosis of growth hormone deficiency (GHD, n = 86) or Turner syndrome (TS, n = 87) in France (1990–2014) and all height measurements of apparently healthy children (referents, n = 923). We modeled the individual height growth curves by applying non-linear mixed models for each new measurement of each child. The resulting growth parameters were used in multinomial logistic regression across five pre-defined age ranges from 1 to 12 years to predict abnormal growth trajectories. For the five age-specific predictive models, we studied the discrimination and calibration curves, and retained the risk thresholds that offered a pre-defined specificity of 98%. Using all the available height measurements for cases and referents, we evaluated the cumulative diagnostic performance of the algorithm for detecting GHD or TS and the theoretical reduction in time to diagnosis. We evaluated these models internally using 5-fold cross-validation and externally from a regional sample of children with GHD (n = 77) or TS (n = 40) and a national sample of apparently healthy children (n = 5,755). The five age-specific predictive models had good discrimination (high AUROC range 0.87-0.99) and good calibration. Internal evaluations showed stable results. External evaluation revealed a cumulative sensitivity and specificity of 84.6% (95% CI 76.8-90.6) and 94.3% (93.6-94.9). The median theoretical reduction in time to diagnosis was 2.0 years (interquartile range 0.6-3.8): 1.6 years (0.5-2.8) for GHD and 3.0 years (1.0-5.4) for TS. To conclude, we developed and internally and externally evaluated an AI algorithm with high diagnostic performance for the early detection of GHD and TS. A refinement of the algorithm to include other target conditions and further external evaluation in other countries is needed before implementation in daily practice.Author summary: Our goal was to improve growth monitoring in children by developing and evaluating an artificial intelligence (AI) algorithm that can detect abnormal growth. We used pre-diagnosis height measurements for children with a diagnosis of growth hormone deficiency (GHD, n = 86) or Turner syndrome (TS, n = 87) and all height measurements of apparently healthy children (referents, n = 923). We modeled the individual height growth curves by applying non-linear mixed models for each new measurement of each child. The resulting growth parameters were used in multinomial logistic regression across five pre-defined age ranges from 1 to 12 years to predict abnormal growth trajectories. For these age-specific predictive models, we achieved good discrimination and calibration, and retained the risk thresholds that offered a pre-defined specificity of 98%. We evaluated these age-specific models internally using 5-fold cross-validation and externally from a regional sample of children with GHD (n = 77) or TS (n = 40) and a national sample of apparently healthy children (n = 5,755). External evaluation revealed cumulative sensitivity and specificity of 84.6% (95% CI 76.8-90.6) and 94.3% (93.6-94.9). The median theoretical reduction in time to diagnosis was 2.0 years (interquartile range 0.6-3.8): 1.6 years (0.5-2.8) for GHD and 3.0 years (1.0-5.4) for TS. A refinement to include other target conditions is needed before implementation in daily practice.