title: Skrr: Skip and Re-use Text Encoder Layers for Memory Efficient
    Text-to-Image Generation
summary: Large-scale text encoders in text-to-image (T2I) diffusion models have
  demonstrated exceptional performance in generating high-quality images from
  textual prompts. Unlike denoising modules that rely on multiple iterative
  steps, text encoders require only a single forward pass to produce text
  embeddings. However, despite their minimal contribution to total inference time
  and floating-point operations (FLOPs), text encoders demand significantly
  higher memory usage, up to eight times more than denoising modules. To address
  this inefficiency, we propose Skip and Re-use layers (Skrr), a simple yet
  effective pruning strategy specifically designed for text encoders in T2I
  diffusion models. Skrr exploits the inherent redundancy in transformer blocks
  by selectively skipping or reusing certain layers in a manner tailored for T2I
  tasks, thereby reducing memory consumption without compromising performance.
  Extensive experiments demonstrate that Skrr maintains image quality comparable
  to the original model even under high sparsity levels, outperforming existing
  blockwise pruning methods. Furthermore, Skrr achieves state-of-the-art memory
  efficiency while preserving performance across multiple evaluation metrics,
  including the FID, CLIP, DreamSim, and GenEval scores.
paper_authors._id: 67aec0a203bf3301ec29ac39
paper_authors.name: Hoigi Seo
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-14T08:01:10.420Z
paper_authors._id: 633e6f07309a99325095dd42
paper_authors.avatarUrl: /avatars/57b91a488ac1745b3c0509c04eb6ad93.svg
paper_authors.fullname: Hoigi Seo
paper_authors.type: user
paper_authors.user: Agorium
paper_authors._id: 67aec0a203bf3301ec29ac3a
paper_authors.name: Wongi Jeong
paper_authors._id: 67aec0a203bf3301ec29ac3b
paper_authors.name: Jae-sun Seo
paper_authors._id: 67aec0a203bf3301ec29ac3c
paper_authors.name: Se Young Chun
paper_title: Skrr: Skip and Re-use Text Encoder Layers for Memory Efficient
    Text-to-Image Generation
paper_summary: Large-scale text encoders in text-to-image (T2I) diffusion models have
  demonstrated exceptional performance in generating high-quality images from
  textual prompts. Unlike denoising modules that rely on multiple iterative
  steps, text encoders require only a single forward pass to produce text
  embeddings. However, despite their minimal contribution to total inference time
  and floating-point operations (FLOPs), text encoders demand significantly
  higher memory usage, up to eight times more than denoising modules. To address
  this inefficiency, we propose Skip and Re-use layers (Skrr), a simple yet
  effective pruning strategy specifically designed for text encoders in T2I
  diffusion models. Skrr exploits the inherent redundancy in transformer blocks
  by selectively skipping or reusing certain layers in a manner tailored for T2I
  tasks, thereby reducing memory consumption without compromising performance.
  Extensive experiments demonstrate that Skrr maintains image quality comparable
  to the original model even under high sparsity levels, outperforming existing
  blockwise pruning methods. Furthermore, Skrr achieves state-of-the-art memory
  efficiency while preserving performance across multiple evaluation metrics,
  including the FID, CLIP, DreamSim, and GenEval scores.
paper_ai_summary: Skip and Re-use layers (Skrr) enhance memory efficiency in text encoders of T2I diffusion models without affecting image quality.
paper_ai_keywords: large-scale text encoders text-to-image (T2I) diffusion models denoising modules text embeddings floating-point operations (FLOPs) Skip and Re-use layers (Skrr) transformer blocks pruning strategy sparsity levels FID CLIP DreamSim GenEval scores