title: Instruction-Guided Autoregressive Neural Network Parameter Generation
summary: Learning to generate neural network parameters conditioned on task
  descriptions and architecture specifications is pivotal for advancing model
  adaptability and transfer learning. Existing methods especially those based on
  diffusion models suffer from limited scalability to large architectures,
  rigidity in handling varying network depths, and disjointed parameter
  generation that undermines inter-layer coherence. In this work, we propose IGPG
  (Instruction Guided Parameter Generation), an autoregressive framework that
  unifies parameter synthesis across diverse tasks and architectures. IGPG
  leverages a VQ-VAE and an autoregressive model to generate neural network
  parameters, conditioned on task instructions, dataset, and architecture
  details. By autoregressively generating neural network weights' tokens, IGPG
  ensures inter-layer coherence and enables efficient adaptation across models
  and datasets. Operating at the token level, IGPG effectively captures complex
  parameter distributions aggregated from a broad spectrum of pretrained models.
  Extensive experiments on multiple vision datasets demonstrate that IGPG
  consolidates diverse pretrained models into a single, flexible generative
  framework. The synthesized parameters achieve competitive or superior
  performance relative to state-of-the-art methods, especially in terms of
  scalability and efficiency when applied to large architectures. These results
  underscore ICPG potential as a powerful tool for pretrained weight retrieval,
  model selection, and rapid task-specific fine-tuning.
paper_authors._id: 67ef5af0724d484dd41afe5c
paper_authors.name: Soro Bedionita
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-04T07:51:20.621Z
paper_authors._id: 66189b980da4c017c401fb5d
paper_authors.avatarUrl: /avatars/d5184741e4a333435022bcb9a4a9b9d8.svg
paper_authors.fullname: soro bedio
paper_authors.type: user
paper_authors.user: bedio
paper_authors._id: 67ef5af0724d484dd41afe5d
paper_authors.name: Bruno Andreis
paper_authors._id: 67ef5af0724d484dd41afe5e
paper_authors.name: Song Chong
paper_authors._id: 67ef5af0724d484dd41afe5f
paper_authors.name: Sung Ju Hwang
paper_title: Instruction-Guided Autoregressive Neural Network Parameter Generation
paper_summary: Learning to generate neural network parameters conditioned on task
  descriptions and architecture specifications is pivotal for advancing model
  adaptability and transfer learning. Existing methods especially those based on
  diffusion models suffer from limited scalability to large architectures,
  rigidity in handling varying network depths, and disjointed parameter
  generation that undermines inter-layer coherence. In this work, we propose IGPG
  (Instruction Guided Parameter Generation), an autoregressive framework that
  unifies parameter synthesis across diverse tasks and architectures. IGPG
  leverages a VQ-VAE and an autoregressive model to generate neural network
  parameters, conditioned on task instructions, dataset, and architecture
  details. By autoregressively generating neural network weights' tokens, IGPG
  ensures inter-layer coherence and enables efficient adaptation across models
  and datasets. Operating at the token level, IGPG effectively captures complex
  parameter distributions aggregated from a broad spectrum of pretrained models.
  Extensive experiments on multiple vision datasets demonstrate that IGPG
  consolidates diverse pretrained models into a single, flexible generative
  framework. The synthesized parameters achieve competitive or superior
  performance relative to state-of-the-art methods, especially in terms of
  scalability and efficiency when applied to large architectures. These results
  underscore ICPG potential as a powerful tool for pretrained weight retrieval,
  model selection, and rapid task-specific fine-tuning.
paper_ai_summary: IGPG, an autoregressive framework using VQ-VAE, generates coherent neural network parameters conditioned on task instructions, achieving competitive performance for scalable and efficient model adaptation and fine-tuning.
paper_ai_keywords: VQ-VAE autoregressive framework parameter synthesis neural network weights' tokens inter-layer coherence pretrained models vision datasets parameter generation task-specific fine-tuning