title: Padding Tone: A Mechanistic Analysis of Padding Tokens in T2I Models
summary: Text-to-image (T2I) diffusion models rely on encoded prompts to guide the
  image generation process. Typically, these prompts are extended to a fixed
  length by adding padding tokens before text encoding. Despite being a default
  practice, the influence of padding tokens on the image generation process has
  not been investigated. In this work, we conduct the first in-depth analysis of
  the role padding tokens play in T2I models. We develop two causal techniques to
  analyze how information is encoded in the representation of tokens across
  different components of the T2I pipeline. Using these techniques, we
  investigate when and how padding tokens impact the image generation process.
  Our findings reveal three distinct scenarios: padding tokens may affect the
  model's output during text encoding, during the diffusion process, or be
  effectively ignored. Moreover, we identify key relationships between these
  scenarios and the model's architecture (cross or self-attention) and its
  training process (frozen or trained text encoder). These insights contribute to
  a deeper understanding of the mechanisms of padding tokens, potentially
  informing future model design and training practices in T2I systems.
paper_authors._id: 67876c0783ec692b4570e0af
paper_authors.name: Michael Toker
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-15T16:45:04.686Z
paper_authors._id: 61c865e3d3702a3bbf50bc04
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61c865e3d3702a3bbf50bc04/eGgcyeUnOHz0hyap5vhvr.jpeg
paper_authors.fullname: Michael Toker
paper_authors.type: user
paper_authors.user: tokeron
paper_authors._id: 67876c0783ec692b4570e0b0
paper_authors.name: Ido Galil
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-15T16:45:37.404Z
paper_authors._id: 66867535e44c9931f329ca43
paper_authors.avatarUrl: /avatars/ec6fc05b9f596f1ce38ec061d3d1d230.svg
paper_authors.fullname: Ido Galil
paper_authors.type: user
paper_authors.user: IdoGalilNvidia
paper_authors._id: 67876c0783ec692b4570e0b1
paper_authors.name: Hadas Orgad
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-15T16:45:44.092Z
paper_authors._id: 60f82853c53e95176a7c6d45
paper_authors.avatarUrl: /avatars/16f6ea944a014af6ebe60499f3460784.svg
paper_authors.fullname: Hadas Orgad
paper_authors.type: user
paper_authors.user: hadasor
paper_authors._id: 67876c0783ec692b4570e0b2
paper_authors.name: Rinon Gal
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-15T16:45:49.560Z
paper_authors._id: 627c1360f19c5eb46d55ba05
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1652712871747-627c1360f19c5eb46d55ba05.jpeg
paper_authors.fullname: Rinon Gal
paper_authors.type: user
paper_authors.user: rinong
paper_authors._id: 67876c0783ec692b4570e0b3
paper_authors.name: Yoad Tewel
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-15T16:46:02.818Z
paper_authors._id: 6201838f305742efd5a6a0e3
paper_authors.avatarUrl: /avatars/4e47ab9ef1a4b9f59756a6d8c90a970f.svg
paper_authors.fullname: Yoad Tewel
paper_authors.type: user
paper_authors.user: YoadTew
paper_authors._id: 67876c0783ec692b4570e0b4
paper_authors.name: Gal Chechik
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-15T16:46:09.446Z
paper_authors._id: 6493393f357b252af72196c5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6493393f357b252af72196c5/EWSy18XRcMRa_4XMM3Fu-.jpeg
paper_authors.fullname: Gal Chechik
paper_authors.type: user
paper_authors.user: galchechik
paper_authors._id: 67876c0783ec692b4570e0b5
paper_authors.name: Yonatan Belinkov
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-15T16:46:15.360Z
paper_authors._id: 614c57f1ee44bcfe57b366d6
paper_authors.avatarUrl: /avatars/186a9aed84681246f48ed2a012c50def.svg
paper_authors.fullname: Yonatan Belinkov
paper_authors.type: user
paper_authors.user: belinkov
paper_title: Padding Tone: A Mechanistic Analysis of Padding Tokens in T2I Models
paper_summary: Text-to-image (T2I) diffusion models rely on encoded prompts to guide the
  image generation process. Typically, these prompts are extended to a fixed
  length by adding padding tokens before text encoding. Despite being a default
  practice, the influence of padding tokens on the image generation process has
  not been investigated. In this work, we conduct the first in-depth analysis of
  the role padding tokens play in T2I models. We develop two causal techniques to
  analyze how information is encoded in the representation of tokens across
  different components of the T2I pipeline. Using these techniques, we
  investigate when and how padding tokens impact the image generation process.
  Our findings reveal three distinct scenarios: padding tokens may affect the
  model's output during text encoding, during the diffusion process, or be
  effectively ignored. Moreover, we identify key relationships between these
  scenarios and the model's architecture (cross or self-attention) and its
  training process (frozen or trained text encoder). These insights contribute to
  a deeper understanding of the mechanisms of padding tokens, potentially
  informing future model design and training practices in T2I systems.
paper_ai_keywords: Text-to-image (T2I) diffusion models encoded prompts padding tokens text encoding causational techniques token representation diffusion process cross-attention self-attention frozen or trained text encoder improvements