Towards 3D Visual Intelligence: Perception, Reconstruction, and Animation

We live in a 3D world. Human vision allows us to effortlessly understand and explore the environment. For example, we perceive the visual world, including scenes, objects, and humans, by estimating their 3D information such as depth and dynamics. We can also reconstruct their appearance and geometry using our reasoning abilities based solely on visual cues. My research aims to bridge the gap between human cognition and machine intelligence through cutting-edge 3D computer vision algorithms like Neural Radiance Fields (NeRF), 3D Gaussian Splatting, and others. Moreover, by understanding the relationship between dynamic changes in geometry/appearance and driving motion, we can further animate reconstructed models using various driving motion signals. In this presentation, I will discuss my latest works, focusing on developing models capable of perceiving, reconstructing, and animating with a solid grounding in 3D knowledge, showcasing these abilities through a blend of theoretical insights and practical demonstrations.