Integrated Synthesis and Optimization of Low-Latency, Multi-Modal Visual Perception Architectures for Adaptive Metropolitan-Scale Traffic Supervision and Anomaly Detection

This paper presents MM-ViPAD (Multi-Modal Visual Perception Architecture for Anomaly Detection), an integrated synthesis and optimization framework for low-latency, adaptive traffic supervision at metropolitan scale. The proposed architecture fuses RGB video, thermal-infrared (IR), and LiDAR point-cloud streams through a novel cross-modal attention transformer, enabling robust vehicle detection, pedestrian tracking, and anomaly identification under diverse environmental conditions including rain, fog, low-light, and high-density traffic scenarios. A hierarchical feature pyramid network (FPN) decoder with depthwise-separable convolutions reduces inference latency to 29.7 ms per frame (33.7 FPS) on an edge-embedded NVIDIA Jetson Orin NX platform while achieving a mean Average Precision (mAP) of 88.6% at IoU threshold 0.5 on the MetroVision-2k benchmark. Five anomaly categories—wrong-way driving, sudden braking, pedestrian intrusion, abandoned object, and traffic congestion—are detected with a mean F1-score of 90.4% and mean AUC-ROC of 0.960. A distributed edge-cloud co-processing protocol sustains near-linear throughput scaling to 128 camera nodes. Experimental validation on three benchmark datasets, including CityFlow-AD and KITTI-360, confirms statistically significant gains over existing single-stream and two-stream baselines.