RSFNet: A retention-based network with spatial–frequency joint enhancement for infrared small target detection

Infrared Small Target Detection (ISTD) is crucial for applications such as traffic monitoring and maritime surveillance. However, it remains highly challenging due to weak target signals and the absence of rich texture information, often resulting in low detection accuracy. Existing deep learning-based ISTD methods typically struggle to balance the trade-off between modeling long-range dependencies and avoiding feature oversmoothing. To this end, we propose RSFNet, a retention-based network with spatial–frequency joint enhancement for ISTD. RSFNet introduces a bidirectional 2D decay–retention attention mechanism into the vision Transformer (ViT) framework, which effectively suppresses background noise while capturing long-range dependencies. In addition, we design a Spatial–Frequency Joint Enhancement Module (SFE) to facilitate the transfer of salient target features from the encoder to the decoder. SFE integrates spatial and frequency domain features to facilitate global–local information interaction. Extensive experiments conducted on multiple publicly available ISTD datasets demonstrate that RSFNet significantly outperforms state-of-the-art (SOTA) methods in both detection accuracy and training efficiency, with a nearly 32% reduction in training time.