title: GaussianFlow: Splatting Gaussian Dynamics for 4D Content Creation
summary: Creating 4D fields of Gaussian Splatting from images or videos is a
  challenging task due to its under-constrained nature. While the optimization
  can draw photometric reference from the input videos or be regulated by
  generative models, directly supervising Gaussian motions remains underexplored.
  In this paper, we introduce a novel concept, Gaussian flow, which connects the
  dynamics of 3D Gaussians and pixel velocities between consecutive frames. The
  Gaussian flow can be efficiently obtained by splatting Gaussian dynamics into
  the image space. This differentiable process enables direct dynamic supervision
  from optical flow. Our method significantly benefits 4D dynamic content
  generation and 4D novel view synthesis with Gaussian Splatting, especially for
  contents with rich motions that are hard to be handled by existing methods. The
  common color drifting issue that happens in 4D generation is also resolved with
  improved Guassian dynamics. Superior visual quality on extensive experiments
  demonstrates our method's effectiveness. Quantitative and qualitative
  evaluations show that our method achieves state-of-the-art results on both
  tasks of 4D generation and 4D novel view synthesis. Project page:
  https://zerg-overmind.github.io/GaussianFlow.github.io/
paper_authors._id: 65fa6512af74a079c161ecab
paper_authors.name: Quankai Gao
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-03-20T04:25:42.331Z
paper_authors._id: 64c12bb1f149752445c39472
paper_authors.avatarUrl: /avatars/8a534f5b850065f72dbe2e036181f7a9.svg
paper_authors.fullname: Quankai Gao
paper_authors.type: user
paper_authors.user: Quankai
paper_authors._id: 65fa6512af74a079c161ecac
paper_authors.name: Qiangeng Xu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-21T08:27:10.225Z
paper_authors._id: 64c1a214c84cd12dd79c1530
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/CSK0AyQhXs3Izrnt9s6_l.png
paper_authors.fullname: qiangeng xu
paper_authors.type: user
paper_authors.user: xharlie
paper_authors._id: 65fa6512af74a079c161ecad
paper_authors.name: Zhe Cao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-21T12:52:42.590Z
paper_authors._id: 6526b4486e00e9a9d6552c9a
paper_authors.avatarUrl: /avatars/d00d93611610432ceee7b976155e013e.svg
paper_authors.fullname: Zhe Cao
paper_authors.type: user
paper_authors.user: huskyc
paper_authors._id: 65fa6512af74a079c161ecae
paper_authors.name: Ben Mildenhall
paper_authors._id: 65fa6512af74a079c161ecaf
paper_authors.name: Wenchao Ma
paper_authors._id: 65fa6512af74a079c161ecb0
paper_authors.name: Le Chen
paper_authors._id: 65fa6512af74a079c161ecb1
paper_authors.name: Danhang Tang
paper_authors._id: 65fa6512af74a079c161ecb2
paper_authors.name: Ulrich Neumann
paper_title: GaussianFlow: Splatting Gaussian Dynamics for 4D Content Creation
paper_summary: Creating 4D fields of Gaussian Splatting from images or videos is a
  challenging task due to its under-constrained nature. While the optimization
  can draw photometric reference from the input videos or be regulated by
  generative models, directly supervising Gaussian motions remains underexplored.
  In this paper, we introduce a novel concept, Gaussian flow, which connects the
  dynamics of 3D Gaussians and pixel velocities between consecutive frames. The
  Gaussian flow can be efficiently obtained by splatting Gaussian dynamics into
  the image space. This differentiable process enables direct dynamic supervision
  from optical flow. Our method significantly benefits 4D dynamic content
  generation and 4D novel view synthesis with Gaussian Splatting, especially for
  contents with rich motions that are hard to be handled by existing methods. The
  common color drifting issue that happens in 4D generation is also resolved with
  improved Guassian dynamics. Superior visual quality on extensive experiments
  demonstrates our method's effectiveness. Quantitative and qualitative
  evaluations show that our method achieves state-of-the-art results on both
  tasks of 4D generation and 4D novel view synthesis. Project page:
  https://zerg-overmind.github.io/GaussianFlow.github.io/
paper_ai_keywords: Gaussian splatting 4D fields Gaussian flow 3D Gaussians pixel velocities differentiable process optical flow 4D dynamic content generation 4D novel view synthesis color drifting Gaussian dynamics