title: PartCraft: Crafting Creative Objects by Parts
summary: This paper propels creative control in generative visual AI by allowing users
  to "select". Departing from traditional text or sketch-based methods, we for
  the first time allow users to choose visual concepts by parts for their
  creative endeavors. The outcome is fine-grained generation that precisely
  captures selected visual concepts, ensuring a holistically faithful and
  plausible result. To achieve this, we first parse objects into parts through
  unsupervised feature clustering. Then, we encode parts into text tokens and
  introduce an entropy-based normalized attention loss that operates on them.
  This loss design enables our model to learn generic prior topology knowledge
  about object's part composition, and further generalize to novel part
  compositions to ensure the generation looks holistically faithful. Lastly, we
  employ a bottleneck encoder to project the part tokens. This not only enhances
  fidelity but also accelerates learning, by leveraging shared knowledge and
  facilitating information exchange among instances. Visual results in the paper
  and supplementary material showcase the compelling power of PartCraft in
  crafting highly customized, innovative creations, exemplified by the "charming"
  and creative birds. Code is released at https://github.com/kamwoh/partcraft.
paper_authors._id: 668bec5cb37c6ac0b09d9657
paper_authors.name: Kam Woh Ng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-09T10:35:36.484Z
paper_authors._id: 646f9a9d5b8225e31ae5a408
paper_authors.avatarUrl: /avatars/b6210993b51bb650ea8aa8c04e5fc351.svg
paper_authors.fullname: kamwoh
paper_authors.type: user
paper_authors.user: kamwoh
paper_authors._id: 668bec5cb37c6ac0b09d9658
paper_authors.name: Xiatian Zhu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-09T10:35:42.899Z
paper_authors._id: 647ef9e7aa8c04bbf9362a8a
paper_authors.avatarUrl: /avatars/dc23e15acf7ee7f3b9f1a68c2716a66d.svg
paper_authors.fullname: Xiatian Zhu
paper_authors.type: user
paper_authors.user: Xiatian-Zhu
paper_authors._id: 668bec5cb37c6ac0b09d9659
paper_authors.name: Yi-Zhe Song
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-09T10:35:56.664Z
paper_authors._id: 6624bd6f9999cbbc78460744
paper_authors.avatarUrl: /avatars/00264325fdf277e290aac5a1206ba73f.svg
paper_authors.fullname: YIZHE SONG
paper_authors.type: user
paper_authors.user: JackeySong
paper_authors._id: 668bec5cb37c6ac0b09d965a
paper_authors.name: Tao Xiang
paper_title: PartCraft: Crafting Creative Objects by Parts
paper_summary: This paper propels creative control in generative visual AI by allowing users
  to "select". Departing from traditional text or sketch-based methods, we for
  the first time allow users to choose visual concepts by parts for their
  creative endeavors. The outcome is fine-grained generation that precisely
  captures selected visual concepts, ensuring a holistically faithful and
  plausible result. To achieve this, we first parse objects into parts through
  unsupervised feature clustering. Then, we encode parts into text tokens and
  introduce an entropy-based normalized attention loss that operates on them.
  This loss design enables our model to learn generic prior topology knowledge
  about object's part composition, and further generalize to novel part
  compositions to ensure the generation looks holistically faithful. Lastly, we
  employ a bottleneck encoder to project the part tokens. This not only enhances
  fidelity but also accelerates learning, by leveraging shared knowledge and
  facilitating information exchange among instances. Visual results in the paper
  and supplementary material showcase the compelling power of PartCraft in
  crafting highly customized, innovative creations, exemplified by the "charming"
  and creative birds. Code is released at https://github.com/kamwoh/partcraft.
paper_ai_keywords: unsupervised feature clustering entropy-based normalized attention loss bottleneck encoder PartCraft