title: Learning Continuous 3D Words for Text-to-Image Generation
summary: Current controls over diffusion models (e.g., through text or ControlNet) for
  image generation fall short in recognizing abstract, continuous attributes like
  illumination direction or non-rigid shape change. In this paper, we present an
  approach for allowing users of text-to-image models to have fine-grained
  control of several attributes in an image. We do this by engineering special
  sets of input tokens that can be transformed in a continuous manner -- we call
  them Continuous 3D Words. These attributes can, for example, be represented as
  sliders and applied jointly with text prompts for fine-grained control over
  image generation. Given only a single mesh and a rendering engine, we show that
  our approach can be adopted to provide continuous user control over several
  3D-aware attributes, including time-of-day illumination, bird wing orientation,
  dollyzoom effect, and object poses. Our method is capable of conditioning image
  creation with multiple Continuous 3D Words and text descriptions simultaneously
  while adding no overhead to the generative process. Project Page:
  https://ttchengab.github.io/continuous_3d_words
paper_authors._id: 65cc2b53ac07b2595b264f95
paper_authors.name: Ta-Ying Cheng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-15T08:28:53.410Z
paper_authors._id: 6442b024e255a338677d78fa
paper_authors.avatarUrl: /avatars/516769c6c3bc7a0d416924181c6866dc.svg
paper_authors.fullname: Ta-Ying Cheng
paper_authors.type: user
paper_authors.user: chengtim
paper_authors._id: 65cc2b53ac07b2595b264f96
paper_authors.name: Matheus Gadelha
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-14T12:52:23.719Z
paper_authors._id: 6315a9c466062476ae1fa0c7
paper_authors.avatarUrl: /avatars/c59749be7ecee4858c59889fea61c88a.svg
paper_authors.fullname: Matheus
paper_authors.type: user
paper_authors.user: matheusgadelha
paper_authors._id: 65cc2b53ac07b2595b264f97
paper_authors.name: Thibault Groueix
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-15T08:28:56.673Z
paper_authors._id: 6340aad6b8b51e0098d7db14
paper_authors.avatarUrl: /avatars/d3af16ef309dbd6fbe2fea27db68332f.svg
paper_authors.fullname: groueix
paper_authors.type: user
paper_authors.user: groueix
paper_authors._id: 65cc2b53ac07b2595b264f98
paper_authors.name: Matthew Fisher
paper_authors._id: 65cc2b53ac07b2595b264f99
paper_authors.name: Radomir Mech
paper_authors._id: 65cc2b53ac07b2595b264f9a
paper_authors.name: Andrew Markham
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-14T12:52:56.940Z
paper_authors._id: 63fca1aa9df782461c7c6367
paper_authors.avatarUrl: /avatars/d61de110a4e87da80a5ca5820921773a.svg
paper_authors.fullname: Andrew Markham
paper_authors.type: user
paper_authors.user: markham
paper_authors._id: 65cc2b53ac07b2595b264f9b
paper_authors.name: Niki Trigoni
paper_title: Learning Continuous 3D Words for Text-to-Image Generation
paper_summary: Current controls over diffusion models (e.g., through text or ControlNet) for
  image generation fall short in recognizing abstract, continuous attributes like
  illumination direction or non-rigid shape change. In this paper, we present an
  approach for allowing users of text-to-image models to have fine-grained
  control of several attributes in an image. We do this by engineering special
  sets of input tokens that can be transformed in a continuous manner -- we call
  them Continuous 3D Words. These attributes can, for example, be represented as
  sliders and applied jointly with text prompts for fine-grained control over
  image generation. Given only a single mesh and a rendering engine, we show that
  our approach can be adopted to provide continuous user control over several
  3D-aware attributes, including time-of-day illumination, bird wing orientation,
  dollyzoom effect, and object poses. Our method is capable of conditioning image
  creation with multiple Continuous 3D Words and text descriptions simultaneously
  while adding no overhead to the generative process. Project Page:
  https://ttchengab.github.io/continuous_3d_words
paper_ai_summary: Continuous 3D Words enable fine-grained control of 3D attributes in text-to-image models, allowing users to adjust image generation through continuous sliders alongside text prompts.
paper_ai_keywords: Continuous 3D Words text-to-image models multidimensional control time-of-day illumination bird wing orientation dollyzoom effect object poses text prompts rendering engine