title: ToDo: Token Downsampling for Efficient Generation of High-Resolution
    Images
summary: Attention mechanism has been crucial for image diffusion models, however,
  their quadratic computational complexity limits the sizes of images we can
  process within reasonable time and memory constraints. This paper investigates
  the importance of dense attention in generative image models, which often
  contain redundant features, making them suitable for sparser attention
  mechanisms. We propose a novel training-free method ToDo that relies on token
  downsampling of key and value tokens to accelerate Stable Diffusion inference
  by up to 2x for common sizes and up to 4.5x or more for high resolutions like
  2048x2048. We demonstrate that our approach outperforms previous methods in
  balancing efficient throughput and fidelity.
paper_authors._id: 65d758719e01a52135cc2d88
paper_authors.name: Ethan Smith
paper_authors._id: 65d758719e01a52135cc2d89
paper_authors.name: Nayan Saxena
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-23T12:25:33.938Z
paper_authors._id: 63b6de91b004c84aac8b970b
paper_authors.avatarUrl: /avatars/fa27d141497093fe0f2d91bf537fad33.svg
paper_authors.fullname: Nayan Saxena
paper_authors.type: user
paper_authors.user: saxenan3
paper_authors._id: 65d758719e01a52135cc2d8a
paper_authors.name: Aninda Saha
paper_title: ToDo: Token Downsampling for Efficient Generation of High-Resolution
    Images
paper_summary: Attention mechanism has been crucial for image diffusion models, however,
  their quadratic computational complexity limits the sizes of images we can
  process within reasonable time and memory constraints. This paper investigates
  the importance of dense attention in generative image models, which often
  contain redundant features, making them suitable for sparser attention
  mechanisms. We propose a novel training-free method ToDo that relies on token
  downsampling of key and value tokens to accelerate Stable Diffusion inference
  by up to 2x for common sizes and up to 4.5x or more for high resolutions like
  2048x2048. We demonstrate that our approach outperforms previous methods in
  balancing efficient throughput and fidelity.
paper_ai_summary: A novel training-free method accelerates Stable Diffusion inference by downsampling key and value tokens, enhancing throughput and fidelity without additional training.
paper_ai_keywords: attention mechanism quadratic computational complexity dense attention generative image models token downsampling Stable Diffusion efficient throughput fidelity