title: HyperFields: Towards Zero-Shot Generation of NeRFs from Text
summary: We introduce HyperFields, a method for generating text-conditioned Neural
  Radiance Fields (NeRFs) with a single forward pass and (optionally) some
  fine-tuning. Key to our approach are: (i) a dynamic hypernetwork, which learns
  a smooth mapping from text token embeddings to the space of NeRFs; (ii) NeRF
  distillation training, which distills scenes encoded in individual NeRFs into
  one dynamic hypernetwork. These techniques enable a single network to fit over
  a hundred unique scenes. We further demonstrate that HyperFields learns a more
  general map between text and NeRFs, and consequently is capable of predicting
  novel in-distribution and out-of-distribution scenes -- either zero-shot or
  with a few finetuning steps. Finetuning HyperFields benefits from accelerated
  convergence thanks to the learned general map, and is capable of synthesizing
  novel scenes 5 to 10 times faster than existing neural optimization-based
  methods. Our ablation experiments show that both the dynamic architecture and
  NeRF distillation are critical to the expressivity of HyperFields.
paper_authors._id: 653b29afd994e992e27b800d
paper_authors.name: Sudarshan Babu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-27T09:12:43.718Z
paper_authors._id: 6341baf206a304bdc432d28b
paper_authors.avatarUrl: /avatars/31fe822d1cd05d9bd7009b97873cb527.svg
paper_authors.fullname: sudarshan babu
paper_authors.type: user
paper_authors.user: sudarshan94
paper_authors._id: 653b29afd994e992e27b800e
paper_authors.name: Richard Liu
paper_authors._id: 653b29afd994e992e27b800f
paper_authors.name: Avery Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-27T09:15:38.847Z
paper_authors._id: 63433036aa45ed8ecf97da55
paper_authors.avatarUrl: /avatars/63566616130bddfae399ff52d600e9e6.svg
paper_authors.fullname: Avery Zhou
paper_authors.type: user
paper_authors.user: averyzhou
paper_authors._id: 653b29afd994e992e27b8010
paper_authors.name: Michael Maire
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-27T09:15:17.865Z
paper_authors._id: 63dcf657f37111482522cd56
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1675425351343-noauth.png
paper_authors.fullname: Michael Maire
paper_authors.type: user
paper_authors.user: MikeStudios
paper_authors._id: 653b29afd994e992e27b8011
paper_authors.name: Greg Shakhnarovich
paper_authors._id: 653b29afd994e992e27b8012
paper_authors.name: Rana Hanocka
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-27T09:15:28.844Z
paper_authors._id: 630495ffce6b12280b193f30
paper_authors.avatarUrl: /avatars/55b201131fa798c2a33d17dcab72fd1f.svg
paper_authors.fullname: Rana Hanocka
paper_authors.type: user
paper_authors.user: ranahanocka
paper_title: HyperFields: Towards Zero-Shot Generation of NeRFs from Text
paper_summary: We introduce HyperFields, a method for generating text-conditioned Neural
  Radiance Fields (NeRFs) with a single forward pass and (optionally) some
  fine-tuning. Key to our approach are: (i) a dynamic hypernetwork, which learns
  a smooth mapping from text token embeddings to the space of NeRFs; (ii) NeRF
  distillation training, which distills scenes encoded in individual NeRFs into
  one dynamic hypernetwork. These techniques enable a single network to fit over
  a hundred unique scenes. We further demonstrate that HyperFields learns a more
  general map between text and NeRFs, and consequently is capable of predicting
  novel in-distribution and out-of-distribution scenes -- either zero-shot or
  with a few finetuning steps. Finetuning HyperFields benefits from accelerated
  convergence thanks to the learned general map, and is capable of synthesizing
  novel scenes 5 to 10 times faster than existing neural optimization-based
  methods. Our ablation experiments show that both the dynamic architecture and
  NeRF distillation are critical to the expressivity of HyperFields.
paper_ai_summary: HyperFields, a method using a dynamic hypernetwork and NeRF distillation, generates text-conditioned NeRFs efficiently across multiple scenes with potentially fast fine-tuning.
paper_ai_keywords: HyperFields dynamic hypernetwork NeRFs text token embeddings NeRF distillation zero-shot fine-tuning neural optimization-based methods