title: Condition-Aware Neural Network for Controlled Image Generation
summary: We present Condition-Aware Neural Network (CAN), a new method for adding
  control to image generative models. In parallel to prior conditional control
  methods, CAN controls the image generation process by dynamically manipulating
  the weight of the neural network. This is achieved by introducing a
  condition-aware weight generation module that generates conditional weight for
  convolution/linear layers based on the input condition. We test CAN on
  class-conditional image generation on ImageNet and text-to-image generation on
  COCO. CAN consistently delivers significant improvements for diffusion
  transformer models, including DiT and UViT. In particular, CAN combined with
  EfficientViT (CaT) achieves 2.78 FID on ImageNet 512x512, surpassing DiT-XL/2
  while requiring 52x fewer MACs per sampling step.
paper_authors._id: 660b9407b29280b06b39ca8e
paper_authors.name: Han Cai
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-28T11:29:08.907Z
paper_authors._id: 650e2b14c945dfc9386a7e28
paper_authors.avatarUrl: /avatars/0a9be20aa53f4c52a2d1a8b02d4093ea.svg
paper_authors.fullname: Han Cai
paper_authors.type: user
paper_authors.user: han-cai
paper_authors._id: 660b9407b29280b06b39ca8f
paper_authors.name: Muyang Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-15T09:48:01.510Z
paper_authors._id: 63129589bbaa385279d1826e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63129589bbaa385279d1826e/0AUs3q4ngRZ-wXuY1jP9G.jpeg
paper_authors.fullname: Muyang Li
paper_authors.type: user
paper_authors.user: Lmxyy
paper_authors._id: 660b9407b29280b06b39ca90
paper_authors.name: Zhuoyang Zhang
paper_authors._id: 660b9407b29280b06b39ca91
paper_authors.name: Qinsheng Zhang
paper_authors._id: 660b9407b29280b06b39ca92
paper_authors.name: Ming-Yu Liu
paper_authors._id: 660b9407b29280b06b39ca93
paper_authors.name: Song Han
paper_title: Condition-Aware Neural Network for Controlled Image Generation
paper_summary: We present Condition-Aware Neural Network (CAN), a new method for adding
  control to image generative models. In parallel to prior conditional control
  methods, CAN controls the image generation process by dynamically manipulating
  the weight of the neural network. This is achieved by introducing a
  condition-aware weight generation module that generates conditional weight for
  convolution/linear layers based on the input condition. We test CAN on
  class-conditional image generation on ImageNet and text-to-image generation on
  COCO. CAN consistently delivers significant improvements for diffusion
  transformer models, including DiT and UViT. In particular, CAN combined with
  EfficientViT (CaT) achieves 2.78 FID on ImageNet 512x512, surpassing DiT-XL/2
  while requiring 52x fewer MACs per sampling step.
paper_ai_summary: A Condition-Aware Neural Network (CAN) method for image generation dynamically adjusts neural network weights based on input conditions, demonstrating significant improvements for diffusion transformer models and surpassing existing models in terms of FID score and computational efficiency.
paper_ai_keywords: Condition-Aware Neural Network (CAN) condition-aware weight generation module diffusion transformer models DiT UViT EfficientViT (CaT) FID MACs