An integrated multi-person pose estimation and activity recognition technique using 3D dual network

Human pose estimation and detection are critical for understanding human activities in videos and images. This paper presents a novel approach to meet the advanced demands of human–computer interactions and assisted living systems through enhanced human pose estimation and activity recognition. We introduce IMPos-DNet, an innovative technique that integrates multi-person pose estimation and activity recognition using a 3D Dual Convolution Neural Network (CNN) applied to multiview video datasets. Our approach combines top-down and bottom-up models to improve performance. The top-down network focuses on evaluating human joints for each individual, enhancing robustness against inaccurate bounding boxes, while the bottom-up network employs normalized heatmaps based on human detection, improving resilience to scale variation. By synergizing the 3D poses estimated by both networks, IMPos-DNet produces precise final 3D poses. Our research objectives include advancing the accuracy and efficiency of pose estimation and activity recognition, as well as addressing the scarcity of 3D ground-truth data. To this end, we employ a semi-supervised method, broadening the model’s applicability. Comprehensive experiments on three publicly available datasets—Human3.6 M, MuPoTs-3D, and MPI-INF-3DHP—demonstrate the model’s superior accuracy and efficiency. Evaluation results confirm the effectiveness of IMPos-DNet’s individual components, highlighting its potential for reliable human pose estimation and activity recognition.