title: BRAT: Bonus oRthogonAl Token for Architecture Agnostic Textual Inversion
summary: Textual Inversion remains a popular method for personalizing diffusion
  models, in order to teach models new subjects and styles. We note that textual
  inversion has been underexplored using alternatives to the UNet, and experiment
  with textual inversion with a vision transformer. We also seek to optimize
  textual inversion using a strategy that does not require explicit use of the
  UNet and its idiosyncratic layers, so we add bonus tokens and enforce
  orthogonality. We find the use of the bonus token improves adherence to the
  source images and the use of the vision transformer improves adherence to the
  prompt. Code is available at https://github.com/jamesBaker361/tex_inv_plus.
paper_authors._id: 66b9cb73046cedd83757190f
paper_authors.name: James Baker
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-08-12T08:44:37.400Z
paper_authors._id: 637317efbd81fae2b3aa400c
paper_authors.avatarUrl: /avatars/898fa184c2d2fb7479ed097b4ddee5e9.svg
paper_authors.fullname: James Baker
paper_authors.type: user
paper_authors.user: jlbaker361
paper_title: BRAT: Bonus oRthogonAl Token for Architecture Agnostic Textual Inversion
paper_summary: Textual Inversion remains a popular method for personalizing diffusion
  models, in order to teach models new subjects and styles. We note that textual
  inversion has been underexplored using alternatives to the UNet, and experiment
  with textual inversion with a vision transformer. We also seek to optimize
  textual inversion using a strategy that does not require explicit use of the
  UNet and its idiosyncratic layers, so we add bonus tokens and enforce
  orthogonality. We find the use of the bonus token improves adherence to the
  source images and the use of the vision transformer improves adherence to the
  prompt. Code is available at https://github.com/jamesBaker361/tex_inv_plus.
paper_ai_summary: Textual inversion is optimized using vision transformers and bonus tokens to improve adherence to source images and prompts without explicit use of UNet.
paper_ai_keywords: textual inversion UNet vision transformer bonus tokens orthogonality