title: Switti: Designing Scale-Wise Transformers for Text-to-Image Synthesis
summary: This work presents Switti, a scale-wise transformer for text-to-image
  generation. Starting from existing next-scale prediction AR models, we first
  explore them for T2I generation and propose architectural modifications to
  improve their convergence and overall performance. We then observe that
  self-attention maps of our pretrained scale-wise AR model exhibit weak
  dependence on preceding scales. Based on this insight, we propose a non-AR
  counterpart facilitating {sim}11% faster sampling and lower memory usage
  while also achieving slightly better generation quality.Furthermore, we reveal
  that classifier-free guidance at high-resolution scales is often unnecessary
  and can even degrade performance. %may be not only unnecessary but potentially
  detrimental. By disabling guidance at these scales, we achieve an additional
  sampling acceleration of {sim}20% and improve the generation of
  fine-grained details. Extensive human preference studies and automated
  evaluations show that Switti outperforms existing T2I AR models and competes
  with state-of-the-art T2I diffusion models while being up to 7{times}
  faster.
paper_authors._id: 674eb16e236b0a3fa92fdacd
paper_authors.name: Anton Voronov
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-03T13:48:25.474Z
paper_authors._id: 6153161e0f96aa6c4d14cbf0
paper_authors.avatarUrl: /avatars/e708193a493b850cf266ad6fb8a8beb7.svg
paper_authors.fullname: Anton Voronov
paper_authors.type: user
paper_authors.user: AntonVoronov
paper_authors._id: 674eb16e236b0a3fa92fdace
paper_authors.name: Denis Kuznedelev
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-03T09:56:35.421Z
paper_authors._id: 629cf0475a13ba8233dd18c9
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1654452258405-noauth.jpeg
paper_authors.fullname: Denis Kuznedelev
paper_authors.type: user
paper_authors.user: SpiridonSunRotator
paper_authors._id: 674eb16e236b0a3fa92fdacf
paper_authors.name: Mikhail Khoroshikh
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-03T13:48:31.974Z
paper_authors._id: 628b8e83ac304a69264a8c9c
paper_authors.avatarUrl: /avatars/a51b630e8d01224d5d8d5edff0a69d30.svg
paper_authors.fullname: Mikhail Khoroshikh
paper_authors.type: user
paper_authors.user: michellemoorre
paper_authors._id: 674eb16e236b0a3fa92fdad0
paper_authors.name: Valentin Khrulkov
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-03T13:48:37.650Z
paper_authors._id: 624c4c51be275a1baf50a19b
paper_authors.avatarUrl: /avatars/aa527fff70d096955962e1964a6840c8.svg
paper_authors.fullname: Valentin Khrulkov
paper_authors.type: user
paper_authors.user: mathemage
paper_authors._id: 674eb16e236b0a3fa92fdad1
paper_authors.name: Dmitry Baranchuk
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-03T13:48:43.760Z
paper_authors._id: 62b6cc49752323892323bc04
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62b6cc49752323892323bc04/gGBld1KJIP9AIpd81L3PC.jpeg
paper_authors.fullname: Dmitry Baranchuk
paper_authors.type: user
paper_authors.user: dbaranchuk
paper_title: Switti: Designing Scale-Wise Transformers for Text-to-Image Synthesis
paper_summary: This work presents Switti, a scale-wise transformer for text-to-image
  generation. Starting from existing next-scale prediction AR models, we first
  explore them for T2I generation and propose architectural modifications to
  improve their convergence and overall performance. We then observe that
  self-attention maps of our pretrained scale-wise AR model exhibit weak
  dependence on preceding scales. Based on this insight, we propose a non-AR
  counterpart facilitating {sim}11% faster sampling and lower memory usage
  while also achieving slightly better generation quality.Furthermore, we reveal
  that classifier-free guidance at high-resolution scales is often unnecessary
  and can even degrade performance. %may be not only unnecessary but potentially
  detrimental. By disabling guidance at these scales, we achieve an additional
  sampling acceleration of {sim}20% and improve the generation of
  fine-grained details. Extensive human preference studies and automated
  evaluations show that Switti outperforms existing T2I AR models and competes
  with state-of-the-art T2I diffusion models while being up to 7{times}
  faster.
paper_ai_keywords: scale-wise transformer text-to-image next-scale prediction AR models self-attention maps non-AR counterpart classifier-free guidance generation quality human preference studies automated evaluations T2I AR models T2I diffusion models