title: Generate Anything Anywhere in Any Scene
summary: Text-to-image diffusion models have attracted considerable interest due to
  their wide applicability across diverse fields. However, challenges persist in
  creating controllable models for personalized object generation. In this paper,
  we first identify the entanglement issues in existing personalized generative
  models, and then propose a straightforward and efficient data augmentation
  training strategy that guides the diffusion model to focus solely on object
  identity. By inserting the plug-and-play adapter layers from a pre-trained
  controllable diffusion model, our model obtains the ability to control the
  location and size of each generated personalized object. During inference, we
  propose a regionally-guided sampling technique to maintain the quality and
  fidelity of the generated images. Our method achieves comparable or superior
  fidelity for personalized objects, yielding a robust, versatile, and
  controllable text-to-image diffusion model that is capable of generating
  realistic and personalized images. Our approach demonstrates significant
  potential for various applications, such as those in art, entertainment, and
  advertising design.
paper_authors._id: 649e4deded55df9fd2458472
paper_authors.name: Yuheng Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-30T19:53:45.610Z
paper_authors._id: 63043c140547362a22a91137
paper_authors.avatarUrl: /avatars/d817f4e5f65c3fab7cef2fd6a3c72632.svg
paper_authors.fullname: Yuheng Li
paper_authors.type: user
paper_authors.user: Yuheng
paper_authors._id: 649e4deded55df9fd2458473
paper_authors.name: Haotian Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-30T19:53:32.526Z
paper_authors._id: 63898b61ec1f539adc0f4da2
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1674248167280-63898b61ec1f539adc0f4da2.jpeg
paper_authors.fullname: Haotian Liu
paper_authors.type: user
paper_authors.user: liuhaotian
paper_authors._id: 649e4deded55df9fd2458474
paper_authors.name: Yangming Wen
paper_authors._id: 649e4deded55df9fd2458475
paper_authors.name: Yong Jae Lee
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-03T07:27:24.245Z
paper_authors._id: 649f41ee70a478f8b36b2984
paper_authors.avatarUrl: /avatars/1c9a76717a450ac4aeb62a1e823d2e4a.svg
paper_authors.fullname: Yong Jae Lee
paper_authors.type: user
paper_authors.user: yjlee0222
paper_title: Generate Anything Anywhere in Any Scene
paper_summary: Text-to-image diffusion models have attracted considerable interest due to
  their wide applicability across diverse fields. However, challenges persist in
  creating controllable models for personalized object generation. In this paper,
  we first identify the entanglement issues in existing personalized generative
  models, and then propose a straightforward and efficient data augmentation
  training strategy that guides the diffusion model to focus solely on object
  identity. By inserting the plug-and-play adapter layers from a pre-trained
  controllable diffusion model, our model obtains the ability to control the
  location and size of each generated personalized object. During inference, we
  propose a regionally-guided sampling technique to maintain the quality and
  fidelity of the generated images. Our method achieves comparable or superior
  fidelity for personalized objects, yielding a robust, versatile, and
  controllable text-to-image diffusion model that is capable of generating
  realistic and personalized images. Our approach demonstrates significant
  potential for various applications, such as those in art, entertainment, and
  advertising design.
paper_ai_summary: A text-to-image diffusion model is enhanced with adapter layers and regionally-guided sampling to achieve controlled generation of personalized objects with high fidelity.
paper_ai_keywords: diffusion models entanglement issues data augmentation plug-and-play adapter layers regionally-guided sampling object identity object location object size personalized object generation controllable diffusion model realistic images