Proposing a cross-correlational-gated recurrent unit neural network for engine block assembly action recognition

The influence of Artificial Intelligence on improving manufacturing system performance has been studied extensively. However, with the increasing variety of factory processes and advancements in digital technologies, new opportunities have emerged to better support operators. Hybrid production systems require seamless collaboration between workers and machines. Human action recognition (HAR) is crucial for developing intuitive machines and robots, enabling more efficient worker-machine interactions. This study introduces a novel gated recurrent unit (GRU) designed to enhance the implementation of HAR, particularly in tasks such as engine block assembly. Traditional deep models struggle to fully grasp the multi-dimensional nature of video data due to its representation in space and time dimensions. In contrast, leveraging optical flow has shown promising results in video processing. The proposed GRU in this study is capable of learning both spatial–temporal features and optical flow feature. The proposed GRU incorporates a "cross-correlation" operator to capture motion dynamics seamlessly without resorting to expensive optical flow computations in an end-to-end framework. Exhaustive tests were carried out on various datasets and the findings states the model's effectiveness in accurately recognizing actions across various datasets, showcasing its generalizability, robustness and versatility in handling diverse challenges in video analysis. Additionally, a specialized dataset on engine block assembly was collected to assess the model's efficacy in manufacturing contexts, demonstrating that the proposed model is robust and generalizable. Graphical abstract