title: Spacetime Gaussian Feature Splatting for Real-Time Dynamic View
    Synthesis
summary: Novel view synthesis of dynamic scenes has been an intriguing yet challenging
  problem. Despite recent advancements, simultaneously achieving high-resolution
  photorealistic results, real-time rendering, and compact storage remains a
  formidable task. To address these challenges, we propose Spacetime Gaussian
  Feature Splatting as a novel dynamic scene representation, composed of three
  pivotal components. First, we formulate expressive Spacetime Gaussians by
  enhancing 3D Gaussians with temporal opacity and parametric motion/rotation.
  This enables Spacetime Gaussians to capture static, dynamic, as well as
  transient content within a scene. Second, we introduce splatted feature
  rendering, which replaces spherical harmonics with neural features. These
  features facilitate the modeling of view- and time-dependent appearance while
  maintaining small size. Third, we leverage the guidance of training error and
  coarse depth to sample new Gaussians in areas that are challenging to converge
  with existing pipelines. Experiments on several established real-world datasets
  demonstrate that our method achieves state-of-the-art rendering quality and
  speed, while retaining compact storage. At 8K resolution, our lite-version
  model can render at 60 FPS on an Nvidia RTX 4090 GPU.
paper_authors._id: 658e4e8835c41262d6248ed0
paper_authors.name: Zhan Li
paper_authors.status: extracted_infirmed
paper_authors.statusLastChangedAt: 2023-12-29T04:53:34.337Z
paper_authors._id: 658e4e8835c41262d6248ed1
paper_authors.name: Zhang Chen
paper_authors._id: 658e4e8835c41262d6248ed2
paper_authors.name: Zhong Li
paper_authors._id: 658e4e8835c41262d6248ed3
paper_authors.name: Yi Xu
paper_title: Spacetime Gaussian Feature Splatting for Real-Time Dynamic View
    Synthesis
paper_summary: Novel view synthesis of dynamic scenes has been an intriguing yet challenging
  problem. Despite recent advancements, simultaneously achieving high-resolution
  photorealistic results, real-time rendering, and compact storage remains a
  formidable task. To address these challenges, we propose Spacetime Gaussian
  Feature Splatting as a novel dynamic scene representation, composed of three
  pivotal components. First, we formulate expressive Spacetime Gaussians by
  enhancing 3D Gaussians with temporal opacity and parametric motion/rotation.
  This enables Spacetime Gaussians to capture static, dynamic, as well as
  transient content within a scene. Second, we introduce splatted feature
  rendering, which replaces spherical harmonics with neural features. These
  features facilitate the modeling of view- and time-dependent appearance while
  maintaining small size. Third, we leverage the guidance of training error and
  coarse depth to sample new Gaussians in areas that are challenging to converge
  with existing pipelines. Experiments on several established real-world datasets
  demonstrate that our method achieves state-of-the-art rendering quality and
  speed, while retaining compact storage. At 8K resolution, our lite-version
  model can render at 60 FPS on an Nvidia RTX 4090 GPU.
paper_ai_keywords: Spacetime Gaussian Feature Splatting Spacetime Gaussians Spherical harmonics Neural features Guidance of training error Coarse depth Gaussians Real-time rendering High-resolution photorealistic results View-dependent appearance Time-dependent appearance Frame Per Second (FPS) Nvidia RTX 4090 GPU