Dynamic Human-Scene Cooperative Novel View Synthesis Method Based on 3D Gaussian Splatting

Dynamic human-scene cooperative novel view synthesis holds significant application value in fields such as Virtual Reality (VR), Augmented Reality (AR), film production, and digital humans. In Chapter 4, we implemented high-fidelity novel view synthesis of real human body surface details based on Neural Radiance Fields (NeRF). Although the synthesis of dynamic human surface details achieved promising results, the slow inference speed of NeRF and its implicit modeling of continuous space - lacking explicit geometric structures - make it difficult to decouple the human body from the scene. Consequently, NeRF fails to meet the requirements for dynamic human-scene cooperative novel view synthesis. Moreover, the absence of accurate semantic segmentation of humans and scenes in three-dimensional space poses a critical challenge in accurately decomposing dynamic human Gaussians and static scene Gaussians. To address these issues, this chapter proposes an efficient dynamic human-scene cooperative novel view synthesis framework based on the 3D Gaussian Splatting (3DGS) method. The framework standardizes the spatial coordinate systems of the human body and the scene to ensure geometric consistency and employs a triplane representation to reconstruct human Gaussians. Finally, a joint training strategy is adopted to simultaneously optimize the human and scene models. Comparative experiments on publicly available datasets demonstrate that the proposed method effectively corrects Gaussian misalignment caused by geometric coupling between the human body and the scene. This results in more accurate decoupling of the human body and the scene, enabling flexible recombination of human and scene elements without additional training, thereby achieving high-quality dynamic human-scene cooperative novel view synthesis.