Hybrid vision transformer model for defect classification in photovoltaic modules using thermographic imaging: Leveraging self-attention mechanisms for enhanced accuracy

Early fault detection and diagnosis are vital for enhancing the efficiency and reliability of photovoltaic (PV) plants. Infrared (IR) thermography imaging has recently been employed to detect defects in PV modules. However, classifying and understanding the nature of defects remain a significant challenge. To address this issue, this paper develops and compares two approaches: a Vision Transformer (ViT) model and five hybrid ViT models that combine the ViT model with various types of Machine Learning (ML) algorithms. In this context, a database of IR thermographic images of faulty PV modules was constructed at two different locations: The UDES in Tipaza (Algeria), and the University of Jaen (Spain). After several inspections, the most commonly observed faults have been identified as short-circuits, dust accumulation, partial shading, and open circuit. The ViT model achieved an accuracy of 99.30 % with an inference time of 1.51 s, while the hybrid ViT model achieved an accuracy of 98.73 % with an inference time of 1.18 ms. Globally, the developed hybrid models have resulted in exceptionally improved classification performance. Furthermore, this research provided an opportunity to compare the effectiveness of using the transformer model alone versus combining it with ML algorithms.