An effective deep learning algorithm for medical image registration

Image registration is crucial for many medical imaging applications, including longitudinal monitoring and multimodal information fusion. A key challenge is to achieve accurate alignment while strictly preserving topology and invertibility. To address the limitations of traditional penalty-based regularization, which may still permit local folding, this study proposes DTC-Reg, a dynamically learned registration framework that more explicitly enforces diffeomorphic deformation. The framework integrates a homotopy-based control–increment formulation with explicit multiscale geometric constraints. Two parameter-sharing U-Nets first extract multiscale feature pyramids from the input images, after which a symmetric registration module with a sequential temporal cascade network progressively refines the forward and inverse multiscale deformation fields. To further enhance diffeomorphic consistency, this study introduces a Multiscale Folding-aware Deformation Correction (MFDC) module that explicitly detects and geometrically rectifies folding points in the predicted deformation fields. Beyond its integration within DTC-Reg, MFDC can also be readily incorporated into several state-of-the-art registration networks, significantly reducing folding and improving deformation regularity. Extensive experiments on three 3D brain MRI registration tasks demonstrate that the proposed method consistently achieves superior performance over existing approaches in both quantitative and qualitative evaluations.Author summary: Automated medical image registration is crucial for diagnosis but often suffers from unrealistic topological distortions. This study introduces DTC-Reg, a deep learning framework that ensures anatomically plausible alignment by mimicking fluid mechanics. Uniquely, it incorporates a geometric correction mechanism, MFDC, that automatically identifies and repairs topological errors. Experiments on brain MRI datasets demonstrate that DTC-Reg achieves state-of-the-art accuracy while completely eliminating tissue folding. This work provides a mathematically rigorous and reliable solution for automated clinical image analysis.