Instruction-Guided Autoregressive Neural Network Parameter Generation
Learning to generate neural network parameters conditioned on task
descriptions and architecture specifications is pivotal for advancing model
adaptability and transfer learning. Existing methods especially those based on
diffusion models suffer from limited scalability to large architectures,
rigidity in handling varying network depths, and disjointed parameter
generation that undermines inter-layer coherence. In this work, we propose IGPG
(Instruction Guided Parameter Generation), an autoregressive framework that
unifies parameter synthesis across diverse tasks and architectures. IGPG
leverages a VQ-VAE and an autoregressive model to generate neural network
parameters, conditioned on task instructions, dataset, and architecture
details. By autoregressively generating neural network weights' tokens, IGPG
ensures inter-layer coherence and enables efficient adaptation across models
and datasets. Operating at the token level, IGPG effectively captures complex
parameter distributions aggregated from a broad spectrum of pretrained models.
Extensive experiments on multiple vision datasets demonstrate that IGPG
consolidates diverse pretrained models into a single, flexible generative
framework. The synthesized parameters achieve competitive or superior
performance relative to state-of-the-art methods, especially in terms of
scalability and efficiency when applied to large architectures. These results
underscore ICPG potential as a powerful tool for pretrained weight retrieval,
model selection, and rapid task-specific fine-tuning.
67ef5af0724d484dd41afe5c
Soro Bedionita
admin_assigned
2025-04-04T07:51:20.621Z
66189b980da4c017c401fb5d
/avatars/d5184741e4a333435022bcb9a4a9b9d8.svg
soro bedio
user
bedio
67ef5af0724d484dd41afe5d
Bruno Andreis
67ef5af0724d484dd41afe5e
Song Chong
67ef5af0724d484dd41afe5f
Sung Ju Hwang
Instruction-Guided Autoregressive Neural Network Parameter Generation
Learning to generate neural network parameters conditioned on task
descriptions and architecture specifications is pivotal for advancing model
adaptability and transfer learning. Existing methods especially those based on
diffusion models suffer from limited scalability to large architectures,
rigidity in handling varying network depths, and disjointed parameter
generation that undermines inter-layer coherence. In this work, we propose IGPG
(Instruction Guided Parameter Generation), an autoregressive framework that
unifies parameter synthesis across diverse tasks and architectures. IGPG
leverages a VQ-VAE and an autoregressive model to generate neural network
parameters, conditioned on task instructions, dataset, and architecture
details. By autoregressively generating neural network weights' tokens, IGPG
ensures inter-layer coherence and enables efficient adaptation across models
and datasets. Operating at the token level, IGPG effectively captures complex
parameter distributions aggregated from a broad spectrum of pretrained models.
Extensive experiments on multiple vision datasets demonstrate that IGPG
consolidates diverse pretrained models into a single, flexible generative
framework. The synthesized parameters achieve competitive or superior
performance relative to state-of-the-art methods, especially in terms of
scalability and efficiency when applied to large architectures. These results
underscore ICPG potential as a powerful tool for pretrained weight retrieval,
model selection, and rapid task-specific fine-tuning.
IGPG, an autoregressive framework using VQ-VAE, generates coherent neural network parameters conditioned on task instructions, achieving competitive performance for scalable and efficient model adaptation and fine-tuning.
VQ-VAE autoregressive framework parameter synthesis neural network weights' tokens inter-layer coherence pretrained models vision datasets parameter generation task-specific fine-tuning