Skrr: Skip and Re-use Text Encoder Layers for Memory Efficient
  Text-to-Image Generation
Large-scale text encoders in text-to-image (T2I) diffusion models have
demonstrated exceptional performance in generating high-quality images from
textual prompts. Unlike denoising modules that rely on multiple iterative
steps, text encoders require only a single forward pass to produce text
embeddings. However, despite their minimal contribution to total inference time
and floating-point operations (FLOPs), text encoders demand significantly
higher memory usage, up to eight times more than denoising modules. To address
this inefficiency, we propose Skip and Re-use layers (Skrr), a simple yet
effective pruning strategy specifically designed for text encoders in T2I
diffusion models. Skrr exploits the inherent redundancy in transformer blocks
by selectively skipping or reusing certain layers in a manner tailored for T2I
tasks, thereby reducing memory consumption without compromising performance.
Extensive experiments demonstrate that Skrr maintains image quality comparable
to the original model even under high sparsity levels, outperforming existing
blockwise pruning methods. Furthermore, Skrr achieves state-of-the-art memory
efficiency while preserving performance across multiple evaluation metrics,
including the FID, CLIP, DreamSim, and GenEval scores.
67aec0a203bf3301ec29ac39
Hoigi Seo
claimed_verified
2025-02-14T08:01:10.420Z
633e6f07309a99325095dd42
/avatars/57b91a488ac1745b3c0509c04eb6ad93.svg
Hoigi Seo
user
Agorium
67aec0a203bf3301ec29ac3a
Wongi Jeong
claimed_verified
2025-07-23T08:38:36.607Z
63a1684281173de5e4caf63b
/avatars/68ac507ffa3ff7cb6899b8c8d002c01f.svg
wongi
user
ignow
67aec0a203bf3301ec29ac3b
Jae-sun Seo
67aec0a203bf3301ec29ac3c
Se Young Chun
Skrr: Skip and Re-use Text Encoder Layers for Memory Efficient
  Text-to-Image Generation
Large-scale text encoders in text-to-image (T2I) diffusion models have
demonstrated exceptional performance in generating high-quality images from
textual prompts. Unlike denoising modules that rely on multiple iterative
steps, text encoders require only a single forward pass to produce text
embeddings. However, despite their minimal contribution to total inference time
and floating-point operations (FLOPs), text encoders demand significantly
higher memory usage, up to eight times more than denoising modules. To address
this inefficiency, we propose Skip and Re-use layers (Skrr), a simple yet
effective pruning strategy specifically designed for text encoders in T2I
diffusion models. Skrr exploits the inherent redundancy in transformer blocks
by selectively skipping or reusing certain layers in a manner tailored for T2I
tasks, thereby reducing memory consumption without compromising performance.
Extensive experiments demonstrate that Skrr maintains image quality comparable
to the original model even under high sparsity levels, outperforming existing
blockwise pruning methods. Furthermore, Skrr achieves state-of-the-art memory
efficiency while preserving performance across multiple evaluation metrics,
including the FID, CLIP, DreamSim, and GenEval scores.
Skip and Re-use layers (Skrr) enhance memory efficiency in text encoders of T2I diffusion models without affecting image quality.
large-scale text encoders text-to-image (T2I) diffusion models denoising modules text embeddings floating-point operations (FLOPs) Skip and Re-use layers (Skrr) transformer blocks pruning strategy sparsity levels FID CLIP DreamSim GenEval scores