title: Semantic Score Distillation Sampling for Compositional Text-to-3D
    Generation
summary: Generating high-quality 3D assets from textual descriptions remains a pivotal
  challenge in computer graphics and vision research. Due to the scarcity of 3D
  data, state-of-the-art approaches utilize pre-trained 2D diffusion priors,
  optimized through Score Distillation Sampling (SDS). Despite progress, crafting
  complex 3D scenes featuring multiple objects or intricate interactions is still
  difficult. To tackle this, recent methods have incorporated box or layout
  guidance. However, these layout-guided compositional methods often struggle to
  provide fine-grained control, as they are generally coarse and lack
  expressiveness. To overcome these challenges, we introduce a novel SDS
  approach, Semantic Score Distillation Sampling (SemanticSDS), designed to
  effectively improve the expressiveness and accuracy of compositional text-to-3D
  generation. Our approach integrates new semantic embeddings that maintain
  consistency across different rendering views and clearly differentiate between
  various objects and parts. These embeddings are transformed into a semantic
  map, which directs a region-specific SDS process, enabling precise optimization
  and compositional generation. By leveraging explicit semantic guidance, our
  method unlocks the compositional capabilities of existing pre-trained diffusion
  models, thereby achieving superior quality in 3D content generation,
  particularly for complex objects and scenes. Experimental results demonstrate
  that our SemanticSDS framework is highly effective for generating
  state-of-the-art complex 3D content. Code:
  https://github.com/YangLing0818/SemanticSDS-3D
paper_authors._id: 670c7992f6b3c3ad2cbe2f2b
paper_authors.name: Ling Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-14T12:35:15.951Z
paper_authors._id: 64fde4e252e82dd432b74ce9
paper_authors.avatarUrl: /avatars/061a69d858b86d1600be916122cae7fc.svg
paper_authors.fullname: Ling Yang
paper_authors.type: user
paper_authors.user: Lingaaaaaaa
paper_authors._id: 670c7992f6b3c3ad2cbe2f2c
paper_authors.name: Zixiang Zhang
paper_authors._id: 670c7992f6b3c3ad2cbe2f2d
paper_authors.name: Junlin Han
paper_authors._id: 670c7992f6b3c3ad2cbe2f2e
paper_authors.name: Bohan Zeng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-14T12:30:09.672Z
paper_authors._id: 6671214c92412fd4640714eb
paper_authors.avatarUrl: /avatars/48fa84e7bc3bb92ad0192aa26b32de10.svg
paper_authors.fullname: bohan zeng
paper_authors.type: user
paper_authors.user: zbhpku
paper_authors._id: 670c7992f6b3c3ad2cbe2f2f
paper_authors.name: Runjia Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-14T12:29:45.711Z
paper_authors._id: 638e29cf319f9c746b87ad4b
paper_authors.avatarUrl: /avatars/70cac8d47847c389eb0393051a64c4a4.svg
paper_authors.fullname: Runjia Li
paper_authors.type: user
paper_authors.user: liguang0115
paper_authors._id: 670c7992f6b3c3ad2cbe2f30
paper_authors.name: Philip Torr
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-14T12:23:22.107Z
paper_authors._id: 6565ed28a5ec0231cb07225f
paper_authors.avatarUrl: /avatars/7f95bba9aa7811d56eecb380827abfac.svg
paper_authors.fullname: prof philip torr
paper_authors.type: user
paper_authors.user: philiptorr
paper_authors._id: 670c7992f6b3c3ad2cbe2f31
paper_authors.name: Wentao Zhang
paper_title: Semantic Score Distillation Sampling for Compositional Text-to-3D
    Generation
paper_summary: Generating high-quality 3D assets from textual descriptions remains a pivotal
  challenge in computer graphics and vision research. Due to the scarcity of 3D
  data, state-of-the-art approaches utilize pre-trained 2D diffusion priors,
  optimized through Score Distillation Sampling (SDS). Despite progress, crafting
  complex 3D scenes featuring multiple objects or intricate interactions is still
  difficult. To tackle this, recent methods have incorporated box or layout
  guidance. However, these layout-guided compositional methods often struggle to
  provide fine-grained control, as they are generally coarse and lack
  expressiveness. To overcome these challenges, we introduce a novel SDS
  approach, Semantic Score Distillation Sampling (SemanticSDS), designed to
  effectively improve the expressiveness and accuracy of compositional text-to-3D
  generation. Our approach integrates new semantic embeddings that maintain
  consistency across different rendering views and clearly differentiate between
  various objects and parts. These embeddings are transformed into a semantic
  map, which directs a region-specific SDS process, enabling precise optimization
  and compositional generation. By leveraging explicit semantic guidance, our
  method unlocks the compositional capabilities of existing pre-trained diffusion
  models, thereby achieving superior quality in 3D content generation,
  particularly for complex objects and scenes. Experimental results demonstrate
  that our SemanticSDS framework is highly effective for generating
  state-of-the-art complex 3D content. Code:
  https://github.com/YangLing0818/SemanticSDS-3D
paper_ai_keywords: Score Distillation Sampling (SDS) denoising diffusion priors 3D data 2D diffusion priors layout guidance compositional methods semantic embeddings semantic map region-specific SDS complex 3D content generation 3D content generation pre-trained diffusion models