SSR-Encoder: Encoding Selective Subject Representation for
  Subject-Driven Generation
Recent advancements in subject-driven image generation have led to zero-shot
generation, yet precise selection and focus on crucial subject representations
remain challenging. Addressing this, we introduce the SSR-Encoder, a novel
architecture designed for selectively capturing any subject from single or
multiple reference images. It responds to various query modalities including
text and masks, without necessitating test-time fine-tuning. The SSR-Encoder
combines a Token-to-Patch Aligner that aligns query inputs with image patches
and a Detail-Preserving Subject Encoder for extracting and preserving fine
features of the subjects, thereby generating subject embeddings. These
embeddings, used in conjunction with original text embeddings, condition the
generation process. Characterized by its model generalizability and efficiency,
the SSR-Encoder adapts to a range of custom models and control modules.
Enhanced by the Embedding Consistency Regularization Loss for improved
training, our extensive experiments demonstrate its effectiveness in versatile
and high-quality image generation, indicating its broad applicability. Project
page: https://ssr-encoder.github.io
658e62eaac02633c0d34f143
Yuxuan Zhang
claimed_verified
2025-03-11T08:26:16.058Z
636b3f9ce3ad78bc68b67541
/avatars/2b7e745953ae39e01222e99fb63b279e.svg
yuxuan
user
zzyx
658e62eaac02633c0d34f144
Jiaming Liu
claimed_verified
2024-08-19T22:15:33.184Z
637c941588699fba70e29f70
https://cdn-avatars.huggingface.co/v1/production/uploads/637c941588699fba70e29f70/b6G_QZkT-MhE47dx87i0d.png
LIU JIAMING
user
jamesliu1217
658e62eaac02633c0d34f145
Yiren Song
claimed_verified
2025-02-04T09:40:37.053Z
64311a95034ecbefddd141ef
/avatars/b6dc5ca373bedbaa368208517954c375.svg
Yiren Song
user
yiren98
658e62eaac02633c0d34f146
Rui Wang
658e62eaac02633c0d34f147
Hao Tang
658e62eaac02633c0d34f148
Jinpeng Yu
658e62eaac02633c0d34f149
Huaxia Li
658e62eaac02633c0d34f14a
Xu Tang
658e62eaac02633c0d34f14b
Yao Hu
658e62eaac02633c0d34f14c
Han Pan
658e62eaac02633c0d34f14d
Zhongliang Jing
SSR-Encoder: Encoding Selective Subject Representation for
  Subject-Driven Generation
Recent advancements in subject-driven image generation have led to zero-shot
generation, yet precise selection and focus on crucial subject representations
remain challenging. Addressing this, we introduce the SSR-Encoder, a novel
architecture designed for selectively capturing any subject from single or
multiple reference images. It responds to various query modalities including
text and masks, without necessitating test-time fine-tuning. The SSR-Encoder
combines a Token-to-Patch Aligner that aligns query inputs with image patches
and a Detail-Preserving Subject Encoder for extracting and preserving fine
features of the subjects, thereby generating subject embeddings. These
embeddings, used in conjunction with original text embeddings, condition the
generation process. Characterized by its model generalizability and efficiency,
the SSR-Encoder adapts to a range of custom models and control modules.
Enhanced by the Embedding Consistency Regularization Loss for improved
training, our extensive experiments demonstrate its effectiveness in versatile
and high-quality image generation, indicating its broad applicability. Project
page: https://ssr-encoder.github.io
The SSR-Encoder is a novel architecture that selectively captures and focuses on crucial subject representations from reference images, enhancing subject-driven image generation without test-time fine-tuning.
SSR-Encoder Token-to-Patch Aligner Detail-Preserving Subject Encoder subject embeddings Embedding Consistency Regularization Loss