Efficient Training with Denoised Neural Weights
Good weight initialization serves as an effective measure to reduce the
training cost of a deep neural network (DNN) model. The choice of how to
initialize parameters is challenging and may require manual tuning, which can
be time-consuming and prone to human error. To overcome such limitations, this
work takes a novel step towards building a weight generator to synthesize the
neural weights for initialization. We use the image-to-image translation task
with generative adversarial networks (GANs) as an example due to the ease of
collecting model weights spanning a wide range. Specifically, we first collect
a dataset with various image editing concepts and their corresponding trained
weights, which are later used for the training of the weight generator. To
address the different characteristics among layers and the substantial number
of weights to be predicted, we divide the weights into equal-sized blocks and
assign each block an index. Subsequently, a diffusion model is trained with
such a dataset using both text conditions of the concept and the block indexes.
By initializing the image translation model with the denoised weights predicted
by our diffusion model, the training requires only 43.3 seconds. Compared to
training from scratch (i.e., Pix2pix), we achieve a 15x training time
acceleration for a new concept while obtaining even better image generation
quality.
66973d22ff9ad37bc39656b2
Yifan Gong
66973d22ff9ad37bc39656b3
Zheng Zhan
66973d22ff9ad37bc39656b4
Yanyu Li
66973d22ff9ad37bc39656b5
Yerlan Idelbayev
66973d22ff9ad37bc39656b6
Andrey Zharkov
66973d22ff9ad37bc39656b7
Kfir Aberman
66973d22ff9ad37bc39656b8
Sergey Tulyakov
66973d22ff9ad37bc39656b9
Yanzhi Wang
66973d22ff9ad37bc39656ba
Jian Ren
claimed_verified
2025-01-15T08:50:52.358Z
61f19829233c91cbd2f79e70
/avatars/a0735a94542b4f7cda5aed8bc4be0538.svg
Jian Ren
user
alanspike
Efficient Training with Denoised Neural Weights
Good weight initialization serves as an effective measure to reduce the
training cost of a deep neural network (DNN) model. The choice of how to
initialize parameters is challenging and may require manual tuning, which can
be time-consuming and prone to human error. To overcome such limitations, this
work takes a novel step towards building a weight generator to synthesize the
neural weights for initialization. We use the image-to-image translation task
with generative adversarial networks (GANs) as an example due to the ease of
collecting model weights spanning a wide range. Specifically, we first collect
a dataset with various image editing concepts and their corresponding trained
weights, which are later used for the training of the weight generator. To
address the different characteristics among layers and the substantial number
of weights to be predicted, we divide the weights into equal-sized blocks and
assign each block an index. Subsequently, a diffusion model is trained with
such a dataset using both text conditions of the concept and the block indexes.
By initializing the image translation model with the denoised weights predicted
by our diffusion model, the training requires only 43.3 seconds. Compared to
training from scratch (i.e., Pix2pix), we achieve a 15x training time
acceleration for a new concept while obtaining even better image generation
quality.
A weight generator using diffusion models and GANs accelerates training for image translation tasks by generating denoised weights for initialization.
generative adversarial networks diffusion model image translation weight initialization denoised weights image editing model weights block indexes