title: Enhancing Detail Preservation for Customized Text-to-Image Generation: A
    Regularization-Free Approach
summary: Recent text-to-image generation models have demonstrated impressive
  capability of generating text-aligned images with high fidelity. However,
  generating images of novel concept provided by the user input image is still a
  challenging task. To address this problem, researchers have been exploring
  various methods for customizing pre-trained text-to-image generation models.
  Currently, most existing methods for customizing pre-trained text-to-image
  generation models involve the use of regularization techniques to prevent
  over-fitting. While regularization will ease the challenge of customization and
  leads to successful content creation with respect to text guidance, it may
  restrict the model capability, resulting in the loss of detailed information
  and inferior performance. In this work, we propose a novel framework for
  customized text-to-image generation without the use of regularization.
  Specifically, our proposed framework consists of an encoder network and a novel
  sampling method which can tackle the over-fitting problem without the use of
  regularization. With the proposed framework, we are able to customize a
  large-scale text-to-image generation model within half a minute on single GPU,
  with only one image provided by the user. We demonstrate in experiments that
  our proposed framework outperforms existing methods, and preserves more
  fine-grained details.
paper_authors._id: 646d69b61aca2e9d9c708eb8
paper_authors.name: Yufan Zhou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-24T18:18:55.038Z
paper_authors._id: 635c2c2a7a165601151d3f85
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1666984965757-noauth.png
paper_authors.fullname: Yufan Zhou
paper_authors.type: user
paper_authors.user: YfZ
paper_authors._id: 646d69b61aca2e9d9c708eb9
paper_authors.name: Ruiyi Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-03T09:42:53.430Z
paper_authors._id: 64668f982da1abc242355cbb
paper_authors.avatarUrl: /avatars/c9a248b8d70b1a8f7b78265c98690570.svg
paper_authors.fullname: Ruiyi Zhang
paper_authors.type: user
paper_authors.user: zhangry868
paper_authors._id: 646d69b61aca2e9d9c708eba
paper_authors.name: Tong Sun
paper_authors._id: 646d69b61aca2e9d9c708ebb
paper_authors.name: Jinhui Xu
paper_title: Enhancing Detail Preservation for Customized Text-to-Image Generation: A
    Regularization-Free Approach
paper_summary: Recent text-to-image generation models have demonstrated impressive
  capability of generating text-aligned images with high fidelity. However,
  generating images of novel concept provided by the user input image is still a
  challenging task. To address this problem, researchers have been exploring
  various methods for customizing pre-trained text-to-image generation models.
  Currently, most existing methods for customizing pre-trained text-to-image
  generation models involve the use of regularization techniques to prevent
  over-fitting. While regularization will ease the challenge of customization and
  leads to successful content creation with respect to text guidance, it may
  restrict the model capability, resulting in the loss of detailed information
  and inferior performance. In this work, we propose a novel framework for
  customized text-to-image generation without the use of regularization.
  Specifically, our proposed framework consists of an encoder network and a novel
  sampling method which can tackle the over-fitting problem without the use of
  regularization. With the proposed framework, we are able to customize a
  large-scale text-to-image generation model within half a minute on single GPU,
  with only one image provided by the user. We demonstrate in experiments that
  our proposed framework outperforms existing methods, and preserves more
  fine-grained details.
paper_ai_summary: A novel framework for customizing text-to-image generation without regularization demonstrates superior performance and fine detail preservation.
paper_ai_keywords: encoder network sampling method over-fitting problem text-to-image generation large-scale model single GPU user image fine-grained details