GSTAR: Gaussian Surface Tracking and Reconstruction
3D Gaussian Splatting techniques have enabled efficient photo-realistic
rendering of static scenes. Recent works have extended these approaches to
support surface reconstruction and tracking. However, tracking dynamic surfaces
with 3D Gaussians remains challenging due to complex topology changes, such as
surfaces appearing, disappearing, or splitting. To address these challenges, we
propose GSTAR, a novel method that achieves photo-realistic rendering, accurate
surface reconstruction, and reliable 3D tracking for general dynamic scenes
with changing topology. Given multi-view captures as input, GSTAR binds
Gaussians to mesh faces to represent dynamic objects. For surfaces with
consistent topology, GSTAR maintains the mesh topology and tracks the meshes
using Gaussians. In regions where topology changes, GSTAR adaptively unbinds
Gaussians from the mesh, enabling accurate registration and the generation of
new surfaces based on these optimized Gaussians. Additionally, we introduce a
surface-based scene flow method that provides robust initialization for
tracking between frames. Experiments demonstrate that our method effectively
tracks and reconstructs dynamic surfaces, enabling a range of applications. Our
project page with the code release is available at
https://eth-ait.github.io/GSTAR/.
67934e1511eb9c774dd1bfc3
Chengwei Zheng
claimed_verified
2025-01-24T13:30:16.388Z
67936c63ddd1e487c0c6c691
/avatars/6d57469b4afdc8bedffeea9ed5f59dd4.svg
Chengwei Zheng
user
zhengcw18
67934e1511eb9c774dd1bfc4
Lixin Xue
admin_assigned
2025-01-24T10:27:50.665Z
645b95f8438d6cfbe1ae8256
/avatars/ac0ebb0a73569ab063c5b2f28c509d23.svg
Lixin Xue
user
lxxue
67934e1511eb9c774dd1bfc5
Juan Zarate
67934e1511eb9c774dd1bfc6
Jie Song
GSTAR: Gaussian Surface Tracking and Reconstruction
3D Gaussian Splatting techniques have enabled efficient photo-realistic
rendering of static scenes. Recent works have extended these approaches to
support surface reconstruction and tracking. However, tracking dynamic surfaces
with 3D Gaussians remains challenging due to complex topology changes, such as
surfaces appearing, disappearing, or splitting. To address these challenges, we
propose GSTAR, a novel method that achieves photo-realistic rendering, accurate
surface reconstruction, and reliable 3D tracking for general dynamic scenes
with changing topology. Given multi-view captures as input, GSTAR binds
Gaussians to mesh faces to represent dynamic objects. For surfaces with
consistent topology, GSTAR maintains the mesh topology and tracks the meshes
using Gaussians. In regions where topology changes, GSTAR adaptively unbinds
Gaussians from the mesh, enabling accurate registration and the generation of
new surfaces based on these optimized Gaussians. Additionally, we introduce a
surface-based scene flow method that provides robust initialization for
tracking between frames. Experiments demonstrate that our method effectively
tracks and reconstructs dynamic surfaces, enabling a range of applications. Our
project page with the code release is available at
https://eth-ait.github.io/GSTAR/.
GSTAR is a method that efficiently tracks and reconstructs dynamic surfaces with changing topology using 3D Gaussian Splatting and mesh-based adaptation.
3D Gaussian Splatting surface reconstruction 3D tracking dynamic surfaces topology changes mesh faces scene flow