title: SteinDreamer: Variance Reduction for Text-to-3D Score Distillation via
    Stein Identity
summary: Score distillation has emerged as one of the most prevalent approaches for
  text-to-3D asset synthesis. Essentially, score distillation updates 3D
  parameters by lifting and back-propagating scores averaged over different
  views. In this paper, we reveal that the gradient estimation in score
  distillation is inherent to high variance. Through the lens of variance
  reduction, the effectiveness of SDS and VSD can be interpreted as applications
  of various control variates to the Monte Carlo estimator of the distilled
  score. Motivated by this rethinking and based on Stein's identity, we propose a
  more general solution to reduce variance for score distillation, termed Stein
  Score Distillation (SSD). SSD incorporates control variates constructed by
  Stein identity, allowing for arbitrary baseline functions. This enables us to
  include flexible guidance priors and network architectures to explicitly
  optimize for variance reduction. In our experiments, the overall pipeline,
  dubbed SteinDreamer, is implemented by instantiating the control variate with a
  monocular depth estimator. The results suggest that SSD can effectively reduce
  the distillation variance and consistently improve visual quality for both
  object- and scene-level generation. Moreover, we demonstrate that SteinDreamer
  achieves faster convergence than existing methods due to more stable gradient
  updates.
paper_authors._id: 65939dbb1adf6d577e153d16
paper_authors.name: Peihao Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-09T09:28:34.015Z
paper_authors._id: 62c8f1773e28b8ee4cddca09
paper_authors.avatarUrl: /avatars/9edbcc84e4ab2b335011daab8acfdbb8.svg
paper_authors.fullname: Peihao Wang
paper_authors.type: user
paper_authors.user: peihaowang
paper_authors._id: 65939dbb1adf6d577e153d17
paper_authors.name: Zhiwen Fan
paper_authors._id: 65939dbb1adf6d577e153d18
paper_authors.name: Dejia Xu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-03T07:06:12.137Z
paper_authors._id: 62f687a0c58915315c4ff75d
paper_authors.avatarUrl: /avatars/b657180c7666735062782edd4f6a69c9.svg
paper_authors.fullname: Dejia Xu
paper_authors.type: user
paper_authors.user: ir1d
paper_authors._id: 65939dbb1adf6d577e153d19
paper_authors.name: Dilin Wang
paper_authors._id: 65939dbb1adf6d577e153d1a
paper_authors.name: Sreyas Mohan
paper_authors._id: 65939dbb1adf6d577e153d1b
paper_authors.name: Forrest Iandola
paper_authors._id: 65939dbb1adf6d577e153d1c
paper_authors.name: Rakesh Ranjan
paper_authors._id: 65939dbb1adf6d577e153d1d
paper_authors.name: Yilei Li
paper_authors._id: 65939dbb1adf6d577e153d1e
paper_authors.name: Qiang Liu
paper_authors._id: 65939dbb1adf6d577e153d1f
paper_authors.name: Zhangyang Wang
paper_authors._id: 65939dbb1adf6d577e153d20
paper_authors.name: Vikas Chandra
paper_title: SteinDreamer: Variance Reduction for Text-to-3D Score Distillation via
    Stein Identity
paper_summary: Score distillation has emerged as one of the most prevalent approaches for
  text-to-3D asset synthesis. Essentially, score distillation updates 3D
  parameters by lifting and back-propagating scores averaged over different
  views. In this paper, we reveal that the gradient estimation in score
  distillation is inherent to high variance. Through the lens of variance
  reduction, the effectiveness of SDS and VSD can be interpreted as applications
  of various control variates to the Monte Carlo estimator of the distilled
  score. Motivated by this rethinking and based on Stein's identity, we propose a
  more general solution to reduce variance for score distillation, termed Stein
  Score Distillation (SSD). SSD incorporates control variates constructed by
  Stein identity, allowing for arbitrary baseline functions. This enables us to
  include flexible guidance priors and network architectures to explicitly
  optimize for variance reduction. In our experiments, the overall pipeline,
  dubbed SteinDreamer, is implemented by instantiating the control variate with a
  monocular depth estimator. The results suggest that SSD can effectively reduce
  the distillation variance and consistently improve visual quality for both
  object- and scene-level generation. Moreover, we demonstrate that SteinDreamer
  achieves faster convergence than existing methods due to more stable gradient
  updates.
paper_ai_keywords: score distillation 3D asset synthesis gradient estimation variance reduction Monte Carlo estimator distilled Score Stein Score Distillation (SSD) control variates Stein identity baseline functions guidance priors network Architectures SteinDreamer monocular depth estimator visual quality object-level generation scene-level generation faster convergence stable gradient updates