Real-time laser keyhole welding penetration state monitoring based on adaptive fusion images using convolutional neural networks

In laser keyhole welding process, the penetration state is an important index to evaluate the quality of weld seam. In this paper, an innovative deep learning-based monitoring system capable of diagnosing the penetration state in real time is introduced in detail. The built monitoring platform can capture the interaction zone images during the laser welding through a high-speed camera. An adaptive fusion method of adjacent monitoring images is proposed to eliminate the interference of metal vapor plume in the image and highlight the keyhole and molten pool. A deep learning model based on convolutional neural network is established to model the relationship between the interaction zone fusion images and the corresponding penetration states. The constructed training set and validation set can improve the robustness and generalization ability of the built model. The verification results on unfused original images show that the average classification accuracy can reach 98.37%, with a latency about 2.9 ms. Finally, a results-evaluation strategy is proposed to eliminate misclassified results to achieve the state-of-the-art performance. The evaluation results of the entire verification weld seam indicate that the proposed monitoring system can meet the requirements of real-time monitoring with high accuracy and short latency. This study provides a novel and robust method for laser welding penetration state monitoring.