Dual scale light weight cross attention transformer for skin lesion classification

Skin cancer is rapidly growing globally. In the past decade, an automated diagnosis system has been developed using image processing and machine learning. The machine learning methods require hand-crafted features, which may affect performance. Recently, a convolution neural network (CNN) was applied to dermoscopic images to diagnose skin cancer. The CNN improved its performance through its high-dimension feature extraction capability. However, these methods lack global co-relation of the spatial features. In this study, we design a dual-scale lightweight cross-attention vision transformer network (DSCATNet) that provides global attention to high-dimensional spatial features. In the DSCATNet, we extracted features from different patch sizes and performed cross-attention. The attention from different scales improved the spatial features by focusing on the different parts of the skin lesion. Furthermore, we applied a fusion strategy for the different scale spatial features. After that, enhanced features are fed to the lightweight transformer encoder for global attention. We validated the model superiority on the HAM 10000 and PAD datasets. Furthermore, the model’s performance is compared with CNN and ViT-based methods. Our DSCATNet achieved an average kappa and accuracy of 95.84% and 97.80% on the HAM 10000 dataset, respectively. Moreover,the model obtained 94.56% and 95.81% kappa and precision values on the PAD dataset.