Modeling Joint Visual Attention in Naturalistic Dyadic Interactions

Joint visual attention (JVA) provides important insight into how individuals coordinate attention during social interaction. Egocentric eye tracking enables the study of JVA in natural, multi-user settings. This work presents a multi-stage framework to identify and analyze JVA using egocentric video and gaze data. The approach consists of three steps: spatiotemporal tube-based visual similarity, gaze-guided object detection, and attention pattern analysis using the ambient–focal coefficient K. Results show that object-focused collaborative activities exhibit high JVA, with object detection capturing higher joint attention than visual similarity, whereas conversation-based or independent activities show lower and more fragmented joint attention. Analysis of K reveals convergence during shared object interaction and divergence during independent tasks. Overall, the study demonstrates the value of combining object-level semantics and attention dynamics to understand JVA in real-world settings, with implications for psychology, human–computer interaction, and social robotics.