title: CustomNet: Zero-shot Object Customization with Variable-Viewpoints in
    Text-to-Image Diffusion Models
summary: Incorporating a customized object into image generation presents an
  attractive feature in text-to-image generation. However, existing
  optimization-based and encoder-based methods are hindered by drawbacks such as
  time-consuming optimization, insufficient identity preservation, and a
  prevalent copy-pasting effect. To overcome these limitations, we introduce
  CustomNet, a novel object customization approach that explicitly incorporates
  3D novel view synthesis capabilities into the object customization process.
  This integration facilitates the adjustment of spatial position relationships
  and viewpoints, yielding diverse outputs while effectively preserving object
  identity. Moreover, we introduce delicate designs to enable location control
  and flexible background control through textual descriptions or specific
  user-defined images, overcoming the limitations of existing 3D novel view
  synthesis methods. We further leverage a dataset construction pipeline that can
  better handle real-world objects and complex backgrounds. Equipped with these
  designs, our method facilitates zero-shot object customization without
  test-time optimization, offering simultaneous control over the viewpoints,
  location, and background. As a result, our CustomNet ensures enhanced identity
  preservation and generates diverse, harmonious outputs.
paper_authors._id: 654093ad992b1c560ed5184a
paper_authors.name: Ziyang Yuan
paper_authors._id: 654093ad992b1c560ed5184b
paper_authors.name: Mingdeng Cao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-31T11:03:27.750Z
paper_authors._id: 6374a02d0856ac905bfc6113
paper_authors.avatarUrl: /avatars/2cbe75c9cc818a647ca6e416f129c96f.svg
paper_authors.fullname: Mingdeng Cao
paper_authors.type: user
paper_authors.user: Ljzycmd
paper_authors._id: 654093ad992b1c560ed5184c
paper_authors.name: Xintao Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-31T11:03:52.814Z
paper_authors._id: 60e272ca6c78a8c122b12127
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/60e272ca6c78a8c122b12127/xldEGBzGrU-bX6IwAw0Ie.jpeg
paper_authors.fullname: Xintao Wang
paper_authors.type: user
paper_authors.user: Xintao
paper_authors._id: 654093ad992b1c560ed5184d
paper_authors.name: Zhongang Qi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-19T08:40:51.534Z
paper_authors._id: 660cc64f1b14f691990c0ea0
paper_authors.avatarUrl: /avatars/f172d3120b22d745a41cc3f2eb499ce6.svg
paper_authors.fullname: Zhongang Qi
paper_authors.type: user
paper_authors.user: phoenixqza
paper_authors._id: 654093ad992b1c560ed5184e
paper_authors.name: Chun Yuan
paper_authors._id: 654093ad992b1c560ed5184f
paper_authors.name: Ying Shan
paper_title: CustomNet: Zero-shot Object Customization with Variable-Viewpoints in
    Text-to-Image Diffusion Models
paper_summary: Incorporating a customized object into image generation presents an
  attractive feature in text-to-image generation. However, existing
  optimization-based and encoder-based methods are hindered by drawbacks such as
  time-consuming optimization, insufficient identity preservation, and a
  prevalent copy-pasting effect. To overcome these limitations, we introduce
  CustomNet, a novel object customization approach that explicitly incorporates
  3D novel view synthesis capabilities into the object customization process.
  This integration facilitates the adjustment of spatial position relationships
  and viewpoints, yielding diverse outputs while effectively preserving object
  identity. Moreover, we introduce delicate designs to enable location control
  and flexible background control through textual descriptions or specific
  user-defined images, overcoming the limitations of existing 3D novel view
  synthesis methods. We further leverage a dataset construction pipeline that can
  better handle real-world objects and complex backgrounds. Equipped with these
  designs, our method facilitates zero-shot object customization without
  test-time optimization, offering simultaneous control over the viewpoints,
  location, and background. As a result, our CustomNet ensures enhanced identity
  preservation and generates diverse, harmonious outputs.
paper_ai_keywords: CustomNet 3D novel view synthesis object customization process spatial position relationships viewpoints delicate designs textual descriptions user-defined images zero-shot object customization test-time optimization identity preservation harmonious outputs