Strokeformer: A novel deep learning paradigm training transformer-based architecture for stroke prognosis prediction

Stroke, a common neurological disorder, is considered one of the leading causes of death and disability worldwide. Stroke prognosis issues involve using clinical characteristics collected from patients presented in tabular form to determine whether they are suitable for thrombolytic therapy. Transformer-based deep learning methods have achieved state-of-the-art performance in various classification tasks, but flaws still exist in dealing with tabular data. These models and algorithms largely tend to overfit and exhibit performance degeneration on small-scale, class-imbalanced datasets. Medical datasets are typically small and imbalanced due to the scarcity of labelled medical data samples. Therefore, this study proposes a novel stroke prognosis prediction model called Strokeformer to address these issues. Specifically, novel intra- and interfeature interaction modules are designed to capture internal and mutual information among individual features for more effective latent representations. In addition, we explore the possibility of performing the training process by pretraining on large-scale, class-balanced datasets and then fine-tuning on small-scale, class-imbalanced downstream datasets. This pretraining and fine-tuning paradigm is dramatically feasible for preventing overfitting. To verify the effectiveness of the proposed model and training method, experiments are conducted on 20 public datasets from OpenML and two private stroke prognosis datasets provided by Shenzhen Fuyong People’s Hospital and The Affiliated Taizhou People’s Hospital of Nanjing Medical University, China, respectively. The results show that Strokeformer performance significantly outperforms that of other comparison models on the introduced datasets. The principal limitation of the model lies in its lack of interpretability from the clinicians’ perspective. Nevertheless, given that the interpretability of deep learning remains an open challenge, the promising empirical results achieved by Strokeformer on real-world stroke prognosis datasets highlight its potential to assist in clinical decision-making.