FaceChain-SuDe: Building Derived Class to Inherit Category Attributes
  for One-shot Subject-Driven Generation
Subject-driven generation has garnered significant interest recently due to
its ability to personalize text-to-image generation. Typical works focus on
learning the new subject's private attributes. However, an important fact has
not been taken seriously that a subject is not an isolated new concept but
should be a specialization of a certain category in the pre-trained model. This
results in the subject failing to comprehensively inherit the attributes in its
category, causing poor attribute-related generations. In this paper, motivated
by object-oriented programming, we model the subject as a derived class whose
base class is its semantic category. This modeling enables the subject to
inherit public attributes from its category while learning its private
attributes from the user-provided example. Specifically, we propose a
plug-and-play method, Subject-Derived regularization (SuDe). It constructs the
base-derived class modeling by constraining the subject-driven generated images
to semantically belong to the subject's category. Extensive experiments under
three baselines and two backbones on various subjects show that our SuDe
enables imaginative attribute-related generations while maintaining subject
fidelity. Codes will be open sourced soon at FaceChain
(https://github.com/modelscope/facechain).
65efc768424023c1047732ef
Pengchong Qiao
admin_assigned
2024-03-12T12:40:30.481Z
64dc69f61855ce11cd799eab
/avatars/511a490c7dd0bd5fa23d18eaf95881b3.svg
qiao
user
pengchonghercy
65efc768424023c1047732f0
Lei Shang
65efc768424023c1047732f1
Chang Liu
65efc768424023c1047732f2
Baigui Sun
admin_assigned
2024-03-12T12:43:08.153Z
64d9f1409a6a7ae9846ab8d1
/avatars/208b9d05b30f0094c3301936fabf4656.svg
Baigui Sun
user
sunbaigui
65efc768424023c1047732f3
Xiangyang Ji
65efc768424023c1047732f4
Jie Chen
FaceChain-SuDe: Building Derived Class to Inherit Category Attributes
  for One-shot Subject-Driven Generation
Subject-driven generation has garnered significant interest recently due to
its ability to personalize text-to-image generation. Typical works focus on
learning the new subject's private attributes. However, an important fact has
not been taken seriously that a subject is not an isolated new concept but
should be a specialization of a certain category in the pre-trained model. This
results in the subject failing to comprehensively inherit the attributes in its
category, causing poor attribute-related generations. In this paper, motivated
by object-oriented programming, we model the subject as a derived class whose
base class is its semantic category. This modeling enables the subject to
inherit public attributes from its category while learning its private
attributes from the user-provided example. Specifically, we propose a
plug-and-play method, Subject-Derived regularization (SuDe). It constructs the
base-derived class modeling by constraining the subject-driven generated images
to semantically belong to the subject's category. Extensive experiments under
three baselines and two backbones on various subjects show that our SuDe
enables imaginative attribute-related generations while maintaining subject
fidelity. Codes will be open sourced soon at FaceChain
(https://github.com/modelscope/facechain).
A plug-and-play method called Subject-Derived regularization (SuDe) ensures that subject-driven text-to-image generation inherits category attributes while learning subject-specific ones, enhancing attribute-related generation quality.
text-to-image generation object-oriented programming derived class semantic category public attributes private attributes imaginative attribute-related generations subject fidelity Subject-Derived regularization SuDe