title: Toward Guidance-Free AR Visual Generation via Condition Contrastive
    Alignment
summary: Classifier-Free Guidance (CFG) is a critical technique for enhancing the
  sample quality of visual generative models. However, in autoregressive (AR)
  multi-modal generation, CFG introduces design inconsistencies between language
  and visual content, contradicting the design philosophy of unifying different
  modalities for visual AR. Motivated by language model alignment methods, we
  propose Condition Contrastive Alignment (CCA) to facilitate
  guidance-free AR visual generation with high performance and analyze its
  theoretical connection with guided sampling methods. Unlike guidance methods
  that alter the sampling process to achieve the ideal sampling distribution, CCA
  directly fine-tunes pretrained models to fit the same distribution target.
  Experimental results show that CCA can significantly enhance the guidance-free
  performance of all tested models with just one epoch of fine-tuning (sim 1\%
  of pretraining epochs) on the pretraining dataset, on par with guided sampling
  methods. This largely removes the need for guided sampling in AR visual
  generation and cuts the sampling cost by half. Moreover, by adjusting training
  parameters, CCA can achieve trade-offs between sample diversity and fidelity
  similar to CFG. This experimentally confirms the strong theoretical connection
  between language-targeted alignment and visual-targeted guidance methods,
  unifying two previously independent research fields. Code and model weights:
  https://github.com/thu-ml/CCA.
paper_authors._id: 6711f37b3eea86736062288b
paper_authors.name: Huayu Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-18T08:09:41.392Z
paper_authors._id: 65571135bfb62d747abc8129
paper_authors.avatarUrl: /avatars/5f4542daa34597f17e6280b9cce18c91.svg
paper_authors.fullname: Hugging
paper_authors.type: user
paper_authors.user: ChenDRAG
paper_authors._id: 6711f37b3eea86736062288c
paper_authors.name: Hang Su
paper_authors._id: 6711f37b3eea86736062288d
paper_authors.name: Peize Sun
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-18T16:40:52.212Z
paper_authors._id: 640dc9bf8512ec51d7f0ac1a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/640dc9bf8512ec51d7f0ac1a/sT4rdEoQbzfW6D3xDVdqt.jpeg
paper_authors.fullname: peizesun
paper_authors.type: user
paper_authors.user: peizesun
paper_authors._id: 6711f37b3eea86736062288e
paper_authors.name: Jun Zhu
paper_title: Toward Guidance-Free AR Visual Generation via Condition Contrastive
    Alignment
paper_summary: Classifier-Free Guidance (CFG) is a critical technique for enhancing the
  sample quality of visual generative models. However, in autoregressive (AR)
  multi-modal generation, CFG introduces design inconsistencies between language
  and visual content, contradicting the design philosophy of unifying different
  modalities for visual AR. Motivated by language model alignment methods, we
  propose Condition Contrastive Alignment (CCA) to facilitate
  guidance-free AR visual generation with high performance and analyze its
  theoretical connection with guided sampling methods. Unlike guidance methods
  that alter the sampling process to achieve the ideal sampling distribution, CCA
  directly fine-tunes pretrained models to fit the same distribution target.
  Experimental results show that CCA can significantly enhance the guidance-free
  performance of all tested models with just one epoch of fine-tuning (sim 1\%
  of pretraining epochs) on the pretraining dataset, on par with guided sampling
  methods. This largely removes the need for guided sampling in AR visual
  generation and cuts the sampling cost by half. Moreover, by adjusting training
  parameters, CCA can achieve trade-offs between sample diversity and fidelity
  similar to CFG. This experimentally confirms the strong theoretical connection
  between language-targeted alignment and visual-targeted guidance methods,
  unifying two previously independent research fields. Code and model weights:
  https://github.com/thu-ml/CCA.
paper_ai_keywords: Classifier-Free Guidance (CFG) autoregressive (AR) Condition Contrastive Alignments (CCA) sampling process guidance-free AR visual generation sampling cost City strong theoretical connection languages-targeted alignment visual-targeted guidance methods