title: Progressive Rendering Distillation: Adapting Stable Diffusion for
    Instant Text-to-Mesh Generation without 3D Data
summary: It is highly desirable to obtain a model that can generate high-quality 3D
  meshes from text prompts in just seconds. While recent attempts have adapted
  pre-trained text-to-image diffusion models, such as Stable Diffusion (SD), into
  generators of 3D representations (e.g., Triplane), they often suffer from poor
  quality due to the lack of sufficient high-quality 3D training data. Aiming at
  overcoming the data shortage, we propose a novel training scheme, termed as
  Progressive Rendering Distillation (PRD), eliminating the need for 3D
  ground-truths by distilling multi-view diffusion models and adapting SD into a
  native 3D generator. In each iteration of training, PRD uses the U-Net to
  progressively denoise the latent from random noise for a few steps, and in each
  step it decodes the denoised latent into 3D output. Multi-view diffusion
  models, including MVDream and RichDreamer, are used in joint with SD to distill
  text-consistent textures and geometries into the 3D outputs through score
  distillation. Since PRD supports training without 3D ground-truths, we can
  easily scale up the training data and improve generation quality for
  challenging text prompts with creative concepts. Meanwhile, PRD can accelerate
  the inference speed of the generation model in just a few steps. With PRD, we
  train a Triplane generator, namely TriplaneTurbo, which adds only 2.5%
  trainable parameters to adapt SD for Triplane generation. TriplaneTurbo
  outperforms previous text-to-3D generators in both efficiency and quality.
  Specifically, it can produce high-quality 3D meshes in 1.2 seconds and
  generalize well for challenging text input. The code is available at
  https://github.com/theEricMa/TriplaneTurbo.
paper_authors._id: 67eb92defa85fe030e2db9e2
paper_authors.name: Zhiyuan Ma
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-01T08:14:51.486Z
paper_authors._id: 64295d1f4e073875f6a605ac
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64295d1f4e073875f6a605ac/HEK8dluqPUhiARW-0eBZG.jpeg
paper_authors.fullname: Zhiyuan Ma
paper_authors.type: user
paper_authors.user: ZhiyuanthePony
paper_authors._id: 67eb92defa85fe030e2db9e3
paper_authors.name: Xinyue Liang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-01T08:14:58.992Z
paper_authors._id: 672111333ced358bdac2925d
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/qHNDI3zYRkJZmhCHkSZwK.png
paper_authors.fullname: Xinyue Liang
paper_authors.type: user
paper_authors.user: DarklordLeto
paper_authors._id: 67eb92defa85fe030e2db9e4
paper_authors.name: Rongyuan Wu
paper_authors._id: 67eb92defa85fe030e2db9e5
paper_authors.name: Xiangyu Zhu
paper_authors._id: 67eb92defa85fe030e2db9e6
paper_authors.name: Zhen Lei
paper_authors._id: 67eb92defa85fe030e2db9e7
paper_authors.name: Lei Zhang
paper_title: Progressive Rendering Distillation: Adapting Stable Diffusion for
    Instant Text-to-Mesh Generation without 3D Data
paper_summary: It is highly desirable to obtain a model that can generate high-quality 3D
  meshes from text prompts in just seconds. While recent attempts have adapted
  pre-trained text-to-image diffusion models, such as Stable Diffusion (SD), into
  generators of 3D representations (e.g., Triplane), they often suffer from poor
  quality due to the lack of sufficient high-quality 3D training data. Aiming at
  overcoming the data shortage, we propose a novel training scheme, termed as
  Progressive Rendering Distillation (PRD), eliminating the need for 3D
  ground-truths by distilling multi-view diffusion models and adapting SD into a
  native 3D generator. In each iteration of training, PRD uses the U-Net to
  progressively denoise the latent from random noise for a few steps, and in each
  step it decodes the denoised latent into 3D output. Multi-view diffusion
  models, including MVDream and RichDreamer, are used in joint with SD to distill
  text-consistent textures and geometries into the 3D outputs through score
  distillation. Since PRD supports training without 3D ground-truths, we can
  easily scale up the training data and improve generation quality for
  challenging text prompts with creative concepts. Meanwhile, PRD can accelerate
  the inference speed of the generation model in just a few steps. With PRD, we
  train a Triplane generator, namely TriplaneTurbo, which adds only 2.5%
  trainable parameters to adapt SD for Triplane generation. TriplaneTurbo
  outperforms previous text-to-3D generators in both efficiency and quality.
  Specifically, it can produce high-quality 3D meshes in 1.2 seconds and
  generalize well for challenging text input. The code is available at
  https://github.com/theEricMa/TriplaneTurbo.
paper_ai_summary: A novel training scheme, Progressive Rendering Distillation (PRD), enhances text-to-3D generation by distilling multi-view diffusion models into a native 3D generator, overcoming data shortages and improving efficiency.
paper_ai_keywords: diffusion models Triplane Progressive Rendering Distillation PRD U-Net multi-view diffusion models MVDream RichDreamer score distillation TriplaneTurbo text-to-3D generators