SplatFlow: Multi-View Rectified Flow Model for 3D Gaussian Splatting
  Synthesis
Text-based generation and editing of 3D scenes hold significant potential for
streamlining content creation through intuitive user interactions. While recent
advances leverage 3D Gaussian Splatting (3DGS) for high-fidelity and real-time
rendering, existing methods are often specialized and task-focused, lacking a
unified framework for both generation and editing. In this paper, we introduce
SplatFlow, a comprehensive framework that addresses this gap by enabling direct
3DGS generation and editing. SplatFlow comprises two main components: a
multi-view rectified flow (RF) model and a Gaussian Splatting Decoder
(GSDecoder). The multi-view RF model operates in latent space, generating
multi-view images, depths, and camera poses simultaneously, conditioned on text
prompts, thus addressing challenges like diverse scene scales and complex
camera trajectories in real-world settings. Then, the GSDecoder efficiently
translates these latent outputs into 3DGS representations through a
feed-forward 3DGS method. Leveraging training-free inversion and inpainting
techniques, SplatFlow enables seamless 3DGS editing and supports a broad range
of 3D tasks-including object editing, novel view synthesis, and camera pose
estimation-within a unified framework without requiring additional complex
pipelines. We validate SplatFlow's capabilities on the MVImgNet and DL3DV-7K
datasets, demonstrating its versatility and effectiveness in various 3D
generation, editing, and inpainting-based tasks.
6745466156ed6a63d03086b0
Hyojun Go
6745466156ed6a63d03086b1
Byeongjun Park
claimed_verified
2024-11-26T21:21:04.414Z
653929a66da48e0d21e65e17
/avatars/e34ae3411d689b4280ff34c1b680f283.svg
Byeongjun Park
user
byeongjun-park
6745466156ed6a63d03086b2
Jiho Jang
admin_assigned
2024-11-26T23:00:01.326Z
66220b24486de1d8db5f5534
/avatars/0fcc480fc0ec80b18c0ca7c9a7a73c34.svg
Jiho Jang
user
hagiss
6745466156ed6a63d03086b3
Jin-Young Kim
admin_assigned
2024-11-26T22:59:40.535Z
65392959899a7547c963e133
/avatars/1727f93161c1e970e891ce7239222f46.svg
Jin-Young Kim
user
Jiny123
6745466156ed6a63d03086b4
Soonwoo Kwon
admin_assigned
2024-11-26T22:59:34.759Z
65000be11e14749e84e6acac
https://cdn-avatars.huggingface.co/v1/production/uploads/65000be11e14749e84e6acac/dX8e5RjQ6wXN-GWz3u8Xe.png
Soonwoo Kwon
user
snoolimal
6745466156ed6a63d03086b5
Changick Kim
SplatFlow: Multi-View Rectified Flow Model for 3D Gaussian Splatting
  Synthesis
Text-based generation and editing of 3D scenes hold significant potential for
streamlining content creation through intuitive user interactions. While recent
advances leverage 3D Gaussian Splatting (3DGS) for high-fidelity and real-time
rendering, existing methods are often specialized and task-focused, lacking a
unified framework for both generation and editing. In this paper, we introduce
SplatFlow, a comprehensive framework that addresses this gap by enabling direct
3DGS generation and editing. SplatFlow comprises two main components: a
multi-view rectified flow (RF) model and a Gaussian Splatting Decoder
(GSDecoder). The multi-view RF model operates in latent space, generating
multi-view images, depths, and camera poses simultaneously, conditioned on text
prompts, thus addressing challenges like diverse scene scales and complex
camera trajectories in real-world settings. Then, the GSDecoder efficiently
translates these latent outputs into 3DGS representations through a
feed-forward 3DGS method. Leveraging training-free inversion and inpainting
techniques, SplatFlow enables seamless 3DGS editing and supports a broad range
of 3D tasks-including object editing, novel view synthesis, and camera pose
estimation-within a unified framework without requiring additional complex
pipelines. We validate SplatFlow's capabilities on the MVImgNet and DL3DV-7K
datasets, demonstrating its versatility and effectiveness in various 3D
generation, editing, and inpainting-based tasks.
SplatFlow is a unified framework for 3D Gaussian Splatting generation and editing, using multi-view rectified flow and Gaussian Splatting Decoder, demonstrating versatility and effectiveness on various 3D tasks.
3D Gaussian Splatting SplatFlow multi-view rectified flow latent space Gaussian Splatting Decoder feed-forward 3DGS training-free inversion inpainting 3D generation 3D editing novel view synthesis camera pose estimation MVImgNet DL3DV-7K