Deep learning–based region merging with adaptive threshold optimization for building segmentation in remote sensing images

Precise extraction of buildings from high-resolution remote sensing images is essential for urban analysis and land management. However, accurately extracting buildings as a region of interest (ROI) from remote sensing (RS) images remains challenging. This difficulty arises from the spectral similarity of other objects, such as roads, cars, or trees, along with limited information on building boundaries and small buildings. Traditional image segmentation methods often rely on a fixed threshold value, making optimisation difficult in cases of over-segmented regions. As a result, region merging is subsequently performed on the region adjacency graph (RAG). Consequently, building segmentation in RS images becomes problematic and can lead to inaccurate boundary delineation or region classification. To overcome these limitations, we propose a novel segmentation approach that incorporates an adaptive thresholding optimisation technique and a merging criterion (MC) based on deep features extracted via a convolutional neural network (CNN)-based AttentionU-Net architecture. This ensures that merging decisions are guided by intrinsic region-level characteristics and refined through deep feature representations. Beginning with initial segmentation generated by the simple linear iterative clustering (SLIC) algorithm, the AttentionU-Net architecture is applied to high-resolution RS images to extract deep features, respectively. As a result, our approach combines both low and high-level feature information, reducing misalignment during merging and enhancing traditional region merging strategies. To validate this approach, the WHU buildings’ RS images dataset was utilised. Experimental results demonstrate that our approach achieves superior segmentation accuracy in building delineation while eliminating the need for rigid thresholds. Finally, the results were compared with those obtained using the multiresolution segmentation (MRS) algorithm implemented in eCognition software on the same WHU buildings RS images, where our approach performs better. Specifically, the proposed approach attained a higher segmentation accuracy, with an F-measure of 0. 91 and a goodness of segmentation score Gs of 0.92, compared to 0.52 and 0.83, respectively, achieved by the MRS algorithm.