VM-Unet enhanced with multi-scale pyramid feature extraction for segmentation of tibiofemoral joint tissues from knee MRI

In medical imaging diagnosis, accurate segmentation of the knee joint can help doctors better observe and diagnose lesions, thereby improving diagnostic accuracy and treatment effectiveness. Vision Mamba mainly relies on the State Space Model (SSM) for feature modeling, which excels at capturing global contextual information but cannot capture local texture features. Moreover, features of different scales are not effectively integrated, resulting in the model’s weak segmentation ability on small-scale tissues (such as cartilage areas). To this end, this study proposed a novel multi-scale Vision Mamba Unet (VM-Unet) framework named MSPF-VM-Unet to perform the segmentation on the femur, tibia, femoral cartilage, and tibial cartilage in knee MRI images. The proposed MSPF-VM-Unet extends VM-Unet by introducing a designed multi-scale pyramid feature extraction network named MPSK, which synergizes multi-resolution feature extraction with channel-space attention. MPSK network enhances multi-scale local feature extraction through Selective Kernel (SK) convolution and pyramid pooling. The network merges the overall context information extracted by the Vision Mamba encoder to achieve the coordinated optimization of a multi-scale hierarchical feature fusion mechanism and global long-range dependency modeling. The results of the comparative experiments on the OAI-ZIB dataset indicate that MSPF-VM-Unet significantly improves the boundary accuracy and regional consistency of the MRI tibiofemoral joint tissue structure.