title: TokenVerse: Versatile Multi-concept Personalization in Token Modulation
    Space
summary: We present TokenVerse -- a method for multi-concept personalization,
  leveraging a pre-trained text-to-image diffusion model. Our framework can
  disentangle complex visual elements and attributes from as little as a single
  image, while enabling seamless plug-and-play generation of combinations of
  concepts extracted from multiple images. As opposed to existing works,
  TokenVerse can handle multiple images with multiple concepts each, and supports
  a wide-range of concepts, including objects, accessories, materials, pose, and
  lighting. Our work exploits a DiT-based text-to-image model, in which the input
  text affects the generation through both attention and modulation (shift and
  scale). We observe that the modulation space is semantic and enables localized
  control over complex concepts. Building on this insight, we devise an
  optimization-based framework that takes as input an image and a text
  description, and finds for each word a distinct direction in the modulation
  space. These directions can then be used to generate new images that combine
  the learned concepts in a desired configuration. We demonstrate the
  effectiveness of TokenVerse in challenging personalization settings, and
  showcase its advantages over existing methods. project's webpage in
  https://token-verse.github.io/
paper_authors._id: 6790db6588d8a90790d99ed4
paper_authors.name: Daniel Garibi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-22T15:32:53.662Z
paper_authors._id: 65fbf768980d4143b0d3ab52
paper_authors.avatarUrl: /avatars/018a26ce944860e9ab97ca43765e7d76.svg
paper_authors.fullname: Daniel Garibi
paper_authors.type: user
paper_authors.user: garibida
paper_authors._id: 6790db6588d8a90790d99ed5
paper_authors.name: Shahar Yadin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-22T15:33:02.856Z
paper_authors._id: 66ab799e5e4f9699e476856e
paper_authors.avatarUrl: /avatars/942943ed751c9703c34782fd7bd92be2.svg
paper_authors.fullname: Shahar Yadin
paper_authors.type: user
paper_authors.user: shaharyadin
paper_authors._id: 6790db6588d8a90790d99ed6
paper_authors.name: Roni Paiss
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-22T15:33:11.237Z
paper_authors._id: 62cab8aeb189b6164d26bfbe
paper_authors.avatarUrl: /avatars/7ed117e375dd9280a5750a242cae24ca.svg
paper_authors.fullname: Roni Paiss
paper_authors.type: user
paper_authors.user: RoniPaiss
paper_authors._id: 6790db6588d8a90790d99ed7
paper_authors.name: Omer Tov
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-22T15:33:26.476Z
paper_authors._id: 623889093657dc674c30954e
paper_authors.avatarUrl: /avatars/a5a193d231e01448cc78ed5bba52bff5.svg
paper_authors.fullname: Omer Tov
paper_authors.type: user
paper_authors.user: omertov
paper_authors._id: 6790db6588d8a90790d99ed8
paper_authors.name: Shiran Zada
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-22T15:33:37.040Z
paper_authors._id: 63510f62cba4ff2e81cb0492
paper_authors.avatarUrl: /avatars/2375a841491de8f40f66b6d0fb0df7b1.svg
paper_authors.fullname: Shiran Zada
paper_authors.type: user
paper_authors.user: shiranzada
paper_authors._id: 6790db6588d8a90790d99ed9
paper_authors.name: Ariel Ephrat
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-22T15:33:44.386Z
paper_authors._id: 65b6ada5ff5235e6ca9551c9
paper_authors.avatarUrl: /avatars/2c558fd133bc1d0cd1b5d7f39d1968ee.svg
paper_authors.fullname: Ariel Ephrat
paper_authors.type: user
paper_authors.user: arielephrat
paper_authors._id: 6790db6588d8a90790d99eda
paper_authors.name: Tomer Michaeli
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-22T15:33:58.984Z
paper_authors._id: 6457c3f50664b05a5b1d941d
paper_authors.avatarUrl: /avatars/0132be4d068d4b314f6b498b65411461.svg
paper_authors.fullname: Tomer Michaeli
paper_authors.type: user
paper_authors.user: Michato
paper_authors._id: 6790db6588d8a90790d99edb
paper_authors.name: Inbar Mosseri
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-22T15:34:07.262Z
paper_authors._id: 670f75a77bb118f080d29eef
paper_authors.avatarUrl: /avatars/134687f97a476be6db5133e26de83ca5.svg
paper_authors.fullname: Inbar Mosseri
paper_authors.type: user
paper_authors.user: inbarm
paper_authors._id: 6790db6588d8a90790d99edc
paper_authors.name: Tali Dekel
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-22T15:34:15.392Z
paper_authors._id: 631cddec68f7da9ad24f6fc7
paper_authors.avatarUrl: /avatars/7d4f1ce805e5889ca6594bd4a93f2583.svg
paper_authors.fullname: Tali Dekel
paper_authors.type: user
paper_authors.user: talidekel
paper_title: TokenVerse: Versatile Multi-concept Personalization in Token Modulation
    Space
paper_summary: We present TokenVerse -- a method for multi-concept personalization,
  leveraging a pre-trained text-to-image diffusion model. Our framework can
  disentangle complex visual elements and attributes from as little as a single
  image, while enabling seamless plug-and-play generation of combinations of
  concepts extracted from multiple images. As opposed to existing works,
  TokenVerse can handle multiple images with multiple concepts each, and supports
  a wide-range of concepts, including objects, accessories, materials, pose, and
  lighting. Our work exploits a DiT-based text-to-image model, in which the input
  text affects the generation through both attention and modulation (shift and
  scale). We observe that the modulation space is semantic and enables localized
  control over complex concepts. Building on this insight, we devise an
  optimization-based framework that takes as input an image and a text
  description, and finds for each word a distinct direction in the modulation
  space. These directions can then be used to generate new images that combine
  the learned concepts in a desired configuration. We demonstrate the
  effectiveness of TokenVerse in challenging personalization settings, and
  showcase its advantages over existing methods. project's webpage in
  https://token-verse.github.io/
paper_ai_summary: TokenVerse uses a DiT-based text-to-image model to disentangle and personalize complex visual concepts from multiple images using an optimization-based framework.
paper_ai_keywords: DiT-based text-to-image model attention modulation modulation space optimization-based framework