Mask-Prior Knowledge Distillation for Efficient Binary Breast Ultrasound Classification

Background: Binary classification of breast ultrasound (BUS) images into benign and malignant categories is a clinically critical task. Standard knowledge distillation (KD) compresses large teacher models into compact student models but transfers knowledge in a spatially blind manner, leaving ground-truth segmentation masks, routinely available in BUS datasets, entirely unexploited.Objective: This study proposes mask-prior knowledge distillation (MP-KD), a novel distillation framework that integrates ground-truth lesion segmentation masks as spatial priors to improve both classification performance and attention interpretability in a lightweight student model.Methods: A multi-teacher ensemble of EfficientNet-B4 and ResNet50, pre-trained and fine-tuned on the Breast Ultrasound Image (BUSI) dataset (697 images; 487 benign, 210 malignant), generates soft targets and attention guidance distilled into an EfficientNet-B0 student (5.3M parameters). The proposed mask-weighted attention transfer (MWAT) loss combines two components: MWAT-AT, which applies mask-derived spatial weights to the attention transfer loss to amplify distillation signals within lesion regions; and MWAT-MA, an auxiliary mask alignment objective that directly supervises student attention towards the ground-truth mask. Loss weights were selected via grid search (18 combinations) on the validation set. All the experiments were repeated with three random seeds (42, 123, 456) on a stratified 70/10/20 split; performance is reported as mean ± standard deviation on the held-out test set.Results: MP-KD achieves the highest AUC of 0.954 ± 0.004 among all the student configurations, outperforming standard KD (0.944 ± 0.002) and the no-KD baseline (0.949 ± 0.004). MP-KD also yields the highest mask-attention alignment score among all the distillation-based configurations (MAAS = 0.608 ± 0.006), substantially recovering the attention alignment lost during standard KD (0.577) while maintaining superior classification performance. Ablation studies show that MWAT-AT primarily drives accuracy (0.908) and specificity (0.921) improvements, while MWAT-MA yields the highest single-run sensitivity (0.952). The combined MP-KD model provides the most robust performance across all the seeds (MCC = 0.732 ± 0.007, AUC std = 0.004, smallest among all the configurations). Compared to the distilled ViT approach of Zhou and Mosadegh (AUC = 0.972, ~86M parameters), MP-KD achieves competitive performance with 16× fewer parameters.Conclusion: MP-KD demonstrates that ground-truth segmentation masks can be effectively integrated as spatial priors within the knowledge distillation pipeline, improving both the AUC and attention alignment of a compact student model compared to standard KD. The proposed MWAT mechanism offers a practical and interpretable solution for deploying breast cancer classification in resource-constrained clinical environments and is generalizable to other medical imaging tasks where segmentation annotations are available.