title: Instant3D: Fast Text-to-3D with Sparse-View Generation and Large
    Reconstruction Model
summary: Text-to-3D with diffusion models have achieved remarkable progress in recent
  years. However, existing methods either rely on score distillation-based
  optimization which suffer from slow inference, low diversity and Janus
  problems, or are feed-forward methods that generate low quality results due to
  the scarcity of 3D training data. In this paper, we propose Instant3D, a novel
  method that generates high-quality and diverse 3D assets from text prompts in a
  feed-forward manner. We adopt a two-stage paradigm, which first generates a
  sparse set of four structured and consistent views from text in one shot with a
  fine-tuned 2D text-to-image diffusion model, and then directly regresses the
  NeRF from the generated images with a novel transformer-based sparse-view
  reconstructor. Through extensive experiments, we demonstrate that our method
  can generate high-quality, diverse and Janus-free 3D assets within 20 seconds,
  which is two order of magnitude faster than previous optimization-based methods
  that can take 1 to 10 hours. Our project webpage: https://jiahao.ai/instant3d/.
paper_authors._id: 65519d760774258908da059a
paper_authors.name: Jiahao Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-22T13:27:25.219Z
paper_authors._id: 630bffcf8b327c7b8b9dfcce
paper_authors.avatarUrl: /avatars/12691f869f01217f7b92a326c30716b1.svg
paper_authors.fullname: Jiahao Li
paper_authors.type: user
paper_authors.user: 54jiahao
paper_authors._id: 65519d760774258908da059b
paper_authors.name: Hao Tan
paper_authors._id: 65519d760774258908da059c
paper_authors.name: Kai Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-11-14T09:37:57.822Z
paper_authors._id: 6351a882a4845f26d16a28d7
paper_authors.avatarUrl: /avatars/4f8629fbf22faeb6511f4068f2828809.svg
paper_authors.fullname: Kai Zhang
paper_authors.type: user
paper_authors.user: Kai-46
paper_authors._id: 65519d760774258908da059d
paper_authors.name: Zexiang Xu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-11-17T07:41:44.064Z
paper_authors._id: 65565b1b8616d8d740abb976
paper_authors.avatarUrl: /avatars/c245b19ed05292bf1bf4a9d8cf2a409c.svg
paper_authors.fullname: Zexiang Xu
paper_authors.type: user
paper_authors.user: zexiangxu
paper_authors._id: 65519d760774258908da059e
paper_authors.name: Fujun Luan
paper_authors._id: 65519d760774258908da059f
paper_authors.name: Yinghao Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-22T13:26:29.903Z
paper_authors._id: 6555da9adcf410fd0753569c
paper_authors.avatarUrl: /avatars/ac58a796bb54f334fdb475a4b75c4d27.svg
paper_authors.fullname: Yinghao Xu
paper_authors.type: user
paper_authors.user: justimyhxu
paper_authors._id: 65519d760774258908da05a0
paper_authors.name: Yicong Hong
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2023-11-13T04:04:30.241Z
paper_authors._id: 654ca66fe1671abcbc50a6e4
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/654ca66fe1671abcbc50a6e4/VqcnC17YcqC9Hk65_LxtE.jpeg
paper_authors.fullname: Yicong Hong
paper_authors.type: user
paper_authors.user: YicongHong
paper_authors._id: 65519d760774258908da05a1
paper_authors.name: Kalyan Sunkavalli
paper_authors._id: 65519d760774258908da05a2
paper_authors.name: Greg Shakhnarovich
paper_authors._id: 65519d760774258908da05a3
paper_authors.name: Sai Bi
paper_title: Instant3D: Fast Text-to-3D with Sparse-View Generation and Large
    Reconstruction Model
paper_summary: Text-to-3D with diffusion models have achieved remarkable progress in recent
  years. However, existing methods either rely on score distillation-based
  optimization which suffer from slow inference, low diversity and Janus
  problems, or are feed-forward methods that generate low quality results due to
  the scarcity of 3D training data. In this paper, we propose Instant3D, a novel
  method that generates high-quality and diverse 3D assets from text prompts in a
  feed-forward manner. We adopt a two-stage paradigm, which first generates a
  sparse set of four structured and consistent views from text in one shot with a
  fine-tuned 2D text-to-image diffusion model, and then directly regresses the
  NeRF from the generated images with a novel transformer-based sparse-view
  reconstructor. Through extensive experiments, we demonstrate that our method
  can generate high-quality, diverse and Janus-free 3D assets within 20 seconds,
  which is two order of magnitude faster than previous optimization-based methods
  that can take 1 to 10 hours. Our project webpage: https://jiahao.ai/instant3d/.
paper_ai_summary: Instant3D rapidly generates high-quality 3D assets from text using a two-stage approach involving a fine-tuned 2D diffusion model and a transformer-based reconstructor.
paper_ai_keywords: text-to-3D diffusion models score distillation-based optimization Janus problems feed-forward methods sparse set of views 2D text-to-image diffusion model NeRF transformer-based sparse-view reconstructor