Centroid-SAM2: A Minimally Supervised Framework for Automated Tongue Image Segmentation in Traditional Chinese Medicine

ObjectiveTongue image segmentation is fundamental for automated tongue diagnosis in Traditional Chinese Medicine (TCM). Existing deep learning approaches suffer from heavy reliance on manual annotation and limited adaptability to annotation-scarce settings, resulting in suboptimal segmentation performance and poor generalization across diverse clinical environments. This study proposes a novel minimally supervised framework. In this approach, the coarse segmentation stage employs lightweight supervised training with limited annotations, while the fine segmentation stage achieves superior accuracy through fully unsupervised, zero-shot inference leveraging SAM2.MethodsWe developed the Centroid-SAM2 model, a minimally supervised tongue image segmentation framework. The first stage employs a coarse segmentation model fine-tuned on 120 annotated images from an independent public dataset, while the second stage leverages the zero-shot capabilities of SAM2 with no labeled data for training or inference; annotations on the target datasets are used solely for evaluation. The approach integrates DeepLabV3_ResNet50 for initial coarse segmentation, followed by a novel centroid coordinate algorithm based on image moments to automatically extract precise guidance points. These centroid coordinates serve as adaptive prompts for the SAM2 model, enabling effective attention guidance in annotation-free segmentation tasks.ResultsComprehensive experiments across three distinct tongue image datasets demonstrated that Centroid-SAM2 significantly outperformed traditional supervised methods and other minimally supervised approaches. The model achieved exceptional performance with 98.21% mIoU, 98.92% accuracy, 99.73% recall, and 99.32% F1-score on Dataset 1, with consistently high performance maintained across Dataset 2 and Dataset 3. Comparative analysis of different SAM2 model scales revealed that sam2.1-hiera_small provided optimal performance with minimal edge artifacts while maintaining computational efficiency.ConclusionThe Centroid-SAM2 model represents a significant advancement in minimally supervised medical image segmentation, substantially reducing dependency on manual effort by confining labeling requirements to a small independent dataset, while achieving superior accuracy in tongue segmentation tasks. This innovative approach addresses the critical challenge of annotation scarcity in medical imaging, offering significant practical value for both TCM modernization and the broader deployment of automated tongue diagnosis systems in complex clinical environments.