title: GSTAR: Gaussian Surface Tracking and Reconstruction
summary: 3D Gaussian Splatting techniques have enabled efficient photo-realistic
  rendering of static scenes. Recent works have extended these approaches to
  support surface reconstruction and tracking. However, tracking dynamic surfaces
  with 3D Gaussians remains challenging due to complex topology changes, such as
  surfaces appearing, disappearing, or splitting. To address these challenges, we
  propose GSTAR, a novel method that achieves photo-realistic rendering, accurate
  surface reconstruction, and reliable 3D tracking for general dynamic scenes
  with changing topology. Given multi-view captures as input, GSTAR binds
  Gaussians to mesh faces to represent dynamic objects. For surfaces with
  consistent topology, GSTAR maintains the mesh topology and tracks the meshes
  using Gaussians. In regions where topology changes, GSTAR adaptively unbinds
  Gaussians from the mesh, enabling accurate registration and the generation of
  new surfaces based on these optimized Gaussians. Additionally, we introduce a
  surface-based scene flow method that provides robust initialization for
  tracking between frames. Experiments demonstrate that our method effectively
  tracks and reconstructs dynamic surfaces, enabling a range of applications. Our
  project page with the code release is available at
  https://eth-ait.github.io/GSTAR/.
paper_authors._id: 67934e1511eb9c774dd1bfc3
paper_authors.name: Chengwei Zheng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-24T13:30:16.388Z
paper_authors._id: 67936c63ddd1e487c0c6c691
paper_authors.avatarUrl: /avatars/6d57469b4afdc8bedffeea9ed5f59dd4.svg
paper_authors.fullname: Chengwei Zheng
paper_authors.type: user
paper_authors.user: zhengcw18
paper_authors._id: 67934e1511eb9c774dd1bfc4
paper_authors.name: Lixin Xue
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-24T10:27:50.665Z
paper_authors._id: 645b95f8438d6cfbe1ae8256
paper_authors.avatarUrl: /avatars/ac0ebb0a73569ab063c5b2f28c509d23.svg
paper_authors.fullname: Lixin Xue
paper_authors.type: user
paper_authors.user: lxxue
paper_authors._id: 67934e1511eb9c774dd1bfc5
paper_authors.name: Juan Zarate
paper_authors._id: 67934e1511eb9c774dd1bfc6
paper_authors.name: Jie Song
paper_title: GSTAR: Gaussian Surface Tracking and Reconstruction
paper_summary: 3D Gaussian Splatting techniques have enabled efficient photo-realistic
  rendering of static scenes. Recent works have extended these approaches to
  support surface reconstruction and tracking. However, tracking dynamic surfaces
  with 3D Gaussians remains challenging due to complex topology changes, such as
  surfaces appearing, disappearing, or splitting. To address these challenges, we
  propose GSTAR, a novel method that achieves photo-realistic rendering, accurate
  surface reconstruction, and reliable 3D tracking for general dynamic scenes
  with changing topology. Given multi-view captures as input, GSTAR binds
  Gaussians to mesh faces to represent dynamic objects. For surfaces with
  consistent topology, GSTAR maintains the mesh topology and tracks the meshes
  using Gaussians. In regions where topology changes, GSTAR adaptively unbinds
  Gaussians from the mesh, enabling accurate registration and the generation of
  new surfaces based on these optimized Gaussians. Additionally, we introduce a
  surface-based scene flow method that provides robust initialization for
  tracking between frames. Experiments demonstrate that our method effectively
  tracks and reconstructs dynamic surfaces, enabling a range of applications. Our
  project page with the code release is available at
  https://eth-ait.github.io/GSTAR/.
paper_ai_keywords: 3D Gaussian Splatting GSTAR photo-realistic rendering surface reconstruction 3D tracking dynamic scenes topology changes Gaussians mesh faces topology maintenance mesh tracking adaptive unbinding accurate registration new surface generation surface-based scene flow robust initialization Algebraic tracking