title: Efficient Training with Denoised Neural Weights
summary: Good weight initialization serves as an effective measure to reduce the
  training cost of a deep neural network (DNN) model. The choice of how to
  initialize parameters is challenging and may require manual tuning, which can
  be time-consuming and prone to human error. To overcome such limitations, this
  work takes a novel step towards building a weight generator to synthesize the
  neural weights for initialization. We use the image-to-image translation task
  with generative adversarial networks (GANs) as an example due to the ease of
  collecting model weights spanning a wide range. Specifically, we first collect
  a dataset with various image editing concepts and their corresponding trained
  weights, which are later used for the training of the weight generator. To
  address the different characteristics among layers and the substantial number
  of weights to be predicted, we divide the weights into equal-sized blocks and
  assign each block an index. Subsequently, a diffusion model is trained with
  such a dataset using both text conditions of the concept and the block indexes.
  By initializing the image translation model with the denoised weights predicted
  by our diffusion model, the training requires only 43.3 seconds. Compared to
  training from scratch (i.e., Pix2pix), we achieve a 15x training time
  acceleration for a new concept while obtaining even better image generation
  quality.
paper_authors._id: 66973d22ff9ad37bc39656b2
paper_authors.name: Yifan Gong
paper_authors._id: 66973d22ff9ad37bc39656b3
paper_authors.name: Zheng Zhan
paper_authors._id: 66973d22ff9ad37bc39656b4
paper_authors.name: Yanyu Li
paper_authors._id: 66973d22ff9ad37bc39656b5
paper_authors.name: Yerlan Idelbayev
paper_authors._id: 66973d22ff9ad37bc39656b6
paper_authors.name: Andrey Zharkov
paper_authors._id: 66973d22ff9ad37bc39656b7
paper_authors.name: Kfir Aberman
paper_authors._id: 66973d22ff9ad37bc39656b8
paper_authors.name: Sergey Tulyakov
paper_authors._id: 66973d22ff9ad37bc39656b9
paper_authors.name: Yanzhi Wang
paper_authors._id: 66973d22ff9ad37bc39656ba
paper_authors.name: Jian Ren
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-15T08:50:52.358Z
paper_authors._id: 61f19829233c91cbd2f79e70
paper_authors.avatarUrl: /avatars/a0735a94542b4f7cda5aed8bc4be0538.svg
paper_authors.fullname: Jian Ren
paper_authors.type: user
paper_authors.user: alanspike
paper_title: Efficient Training with Denoised Neural Weights
paper_summary: Good weight initialization serves as an effective measure to reduce the
  training cost of a deep neural network (DNN) model. The choice of how to
  initialize parameters is challenging and may require manual tuning, which can
  be time-consuming and prone to human error. To overcome such limitations, this
  work takes a novel step towards building a weight generator to synthesize the
  neural weights for initialization. We use the image-to-image translation task
  with generative adversarial networks (GANs) as an example due to the ease of
  collecting model weights spanning a wide range. Specifically, we first collect
  a dataset with various image editing concepts and their corresponding trained
  weights, which are later used for the training of the weight generator. To
  address the different characteristics among layers and the substantial number
  of weights to be predicted, we divide the weights into equal-sized blocks and
  assign each block an index. Subsequently, a diffusion model is trained with
  such a dataset using both text conditions of the concept and the block indexes.
  By initializing the image translation model with the denoised weights predicted
  by our diffusion model, the training requires only 43.3 seconds. Compared to
  training from scratch (i.e., Pix2pix), we achieve a 15x training time
  acceleration for a new concept while obtaining even better image generation
  quality.
paper_ai_keywords: generative adversarial networks (GANs) diffusion model denoised weights image-to-image translation text conditions image generation quality training from scratch (i.e., Pix2pix)