SelectSeg: Uncertainty-based selective training and prediction for accurate crack segmentation under limited data and noisy annotations

The performance of deep learning models in crack segmentation heavily depends on the availability of large-scale, pixel-wise annotated datasets. However, such annotation is costly to acquire, and can be noisy due to the complexity of crack patterns and the subjectivity of human annotators. To obtain accurate crack segmentation models under noisy annotations, we propose SelectSeg – a four-stage uncertainty-based framework. First, we start with training a deep ensemble of segmentation models to capture the crack prediction uncertainties. Secondly, an uncertainty-based filtering mechanism identifies possibly noisy annotations. Thirdly, semi-supervised learning leverages the information from both reliably annotated data (labeled) and unreliably annotated data (unlabeled) to retrain the segmentation model. Finally, a selective prediction mechanism allows the model to abstain from making predictions on challenging cases, enhancing the overall workflow reliability. Experimental results on real-world crack datasets demonstrate that SelectSeg outperforms existing methods in noisy annotation scenarios. Both selective training and prediction bring significant accuracy improvement.