title: GaussianAnything: Interactive Point Cloud Latent Diffusion for 3D
    Generation
summary: While 3D content generation has advanced significantly, existing methods
  still face challenges with input formats, latent space design, and output
  representations. This paper introduces a novel 3D generation framework that
  addresses these challenges, offering scalable, high-quality 3D generation with
  an interactive Point Cloud-structured Latent space. Our framework employs a
  Variational Autoencoder (VAE) with multi-view posed RGB-D(epth)-N(ormal)
  renderings as input, using a unique latent space design that preserves 3D shape
  information, and incorporates a cascaded latent diffusion model for improved
  shape-texture disentanglement. The proposed method, GaussianAnything, supports
  multi-modal conditional 3D generation, allowing for point cloud, caption, and
  single/multi-view image inputs. Notably, the newly proposed latent space
  naturally enables geometry-texture disentanglement, thus allowing 3D-aware
  editing. Experimental results demonstrate the effectiveness of our approach on
  multiple datasets, outperforming existing methods in both text- and
  image-conditioned 3D generation.
paper_authors._id: 673aa0f7c36fe6d2f0a1a9a9
paper_authors.name: Yushi Lan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-18T07:47:19.142Z
paper_authors._id: 64b55f62e6b0c53f97e64034
paper_authors.avatarUrl: /avatars/d5fc53de37d4fb0b5095a5fb21b43503.svg
paper_authors.fullname: LAN YUSHI
paper_authors.type: user
paper_authors.user: yslan
paper_authors._id: 673aa0f7c36fe6d2f0a1a9aa
paper_authors.name: Shangchen Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-18T07:49:40.488Z
paper_authors._id: 62e57662ae9d3f10acbb1b1b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62e57662ae9d3f10acbb1b1b/lg58jdbNyv6LGH2LFnZDF.png
paper_authors.fullname: Shangchen Zhou
paper_authors.type: user
paper_authors.user: sczhou
paper_authors._id: 673aa0f7c36fe6d2f0a1a9ab
paper_authors.name: Zhaoyang Lyu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-18T07:49:45.706Z
paper_authors._id: 63f2ec797ddf724fbcc75aee
paper_authors.avatarUrl: /avatars/e93432ad11da703d46fe5e594d69f8c0.svg
paper_authors.fullname: Zhaoyang Lyu
paper_authors.type: user
paper_authors.user: ZhaoyangLyu
paper_authors._id: 673aa0f7c36fe6d2f0a1a9ac
paper_authors.name: Fangzhou Hong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-18T07:49:50.863Z
paper_authors._id: 623c530013a63ea865f96c8e
paper_authors.avatarUrl: /avatars/164455a1a94f92b71733fc778c21bd89.svg
paper_authors.fullname: Fangzhou Hong
paper_authors.type: user
paper_authors.user: hongfz16
paper_authors._id: 673aa0f7c36fe6d2f0a1a9ad
paper_authors.name: Shuai Yang
paper_authors._id: 673aa0f7c36fe6d2f0a1a9ae
paper_authors.name: Bo Dai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-18T07:50:09.291Z
paper_authors._id: 665e2bd0fbaa279db369b488
paper_authors.avatarUrl: /avatars/8cbc18d17d2813617976dbe709235799.svg
paper_authors.fullname: Bo Dai
paper_authors.type: user
paper_authors.user: doubling
paper_authors._id: 673aa0f7c36fe6d2f0a1a9af
paper_authors.name: Xingang Pan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-18T07:50:35.048Z
paper_authors._id: 646758d6a9b4610868a138ff
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/646758d6a9b4610868a138ff/fNBlK5g4g7vlPAR1bQimZ.png
paper_authors.fullname: Xingang Pan
paper_authors.type: user
paper_authors.user: Xingang-Pan
paper_authors._id: 673aa0f7c36fe6d2f0a1a9b0
paper_authors.name: Chen Change Loy
paper_title: GaussianAnything: Interactive Point Cloud Latent Diffusion for 3D
    Generation
paper_summary: While 3D content generation has advanced significantly, existing methods
  still face challenges with input formats, latent space design, and output
  representations. This paper introduces a novel 3D generation framework that
  addresses these challenges, offering scalable, high-quality 3D generation with
  an interactive Point Cloud-structured Latent space. Our framework employs a
  Variational Autoencoder (VAE) with multi-view posed RGB-D(epth)-N(ormal)
  renderings as input, using a unique latent space design that preserves 3D shape
  information, and incorporates a cascaded latent diffusion model for improved
  shape-texture disentanglement. The proposed method, GaussianAnything, supports
  multi-modal conditional 3D generation, allowing for point cloud, caption, and
  single/multi-view image inputs. Notably, the newly proposed latent space
  naturally enables geometry-texture disentanglement, thus allowing 3D-aware
  editing. Experimental results demonstrate the effectiveness of our approach on
  multiple datasets, outperforming existing methods in both text- and
  image-conditioned 3D generation.
paper_ai_summary: A novel 3D generation framework using a VAE and cascaded latent diffusion model in a Point Cloud-structured Latent space achieves high-quality 3D generation with multi-modal inputs and disentangled shape-texture editing.
paper_ai_keywords: Variational Autoencoder latent space RGB-D multi-view latent diffusion model shape-texture disentanglement Point Cloud-structured Latent space GaussianAnything 3D-aware editing