title: Dreamer XL: Towards High-Resolution Text-to-3D Generation via Trajectory
    Score Matching
summary: In this work, we propose a novel Trajectory Score Matching (TSM) method that
  aims to solve the pseudo ground truth inconsistency problem caused by the
  accumulated error in Interval Score Matching (ISM) when using the Denoising
  Diffusion Implicit Models (DDIM) inversion process. Unlike ISM which adopts the
  inversion process of DDIM to calculate on a single path, our TSM method
  leverages the inversion process of DDIM to generate two paths from the same
  starting point for calculation. Since both paths start from the same starting
  point, TSM can reduce the accumulated error compared to ISM, thus alleviating
  the problem of pseudo ground truth inconsistency. TSM enhances the stability
  and consistency of the model's generated paths during the distillation process.
  We demonstrate this experimentally and further show that ISM is a special case
  of TSM. Furthermore, to optimize the current multi-stage optimization process
  from high-resolution text to 3D generation, we adopt Stable Diffusion XL for
  guidance. In response to the issues of abnormal replication and splitting
  caused by unstable gradients during the 3D Gaussian splatting process when
  using Stable Diffusion XL, we propose a pixel-by-pixel gradient clipping
  method. Extensive experiments show that our model significantly surpasses the
  state-of-the-art models in terms of visual quality and performance. Code:
  https://github.com/xingy038/Dreamer-XL.
paper_authors._id: 664c09bf877f8871f4422023
paper_authors.name: Xingyu Miao
paper_authors._id: 664c09bf877f8871f4422024
paper_authors.name: Haoran Duan
paper_authors._id: 664c09bf877f8871f4422025
paper_authors.name: Varun Ojha
paper_authors._id: 664c09bf877f8871f4422026
paper_authors.name: Jun Song
paper_authors._id: 664c09bf877f8871f4422027
paper_authors.name: Tejal Shah
paper_authors._id: 664c09bf877f8871f4422028
paper_authors.name: Yang Long
paper_authors._id: 664c09bf877f8871f4422029
paper_authors.name: Rajiv Ranjan
paper_title: Dreamer XL: Towards High-Resolution Text-to-3D Generation via Trajectory
    Score Matching
paper_summary: In this work, we propose a novel Trajectory Score Matching (TSM) method that
  aims to solve the pseudo ground truth inconsistency problem caused by the
  accumulated error in Interval Score Matching (ISM) when using the Denoising
  Diffusion Implicit Models (DDIM) inversion process. Unlike ISM which adopts the
  inversion process of DDIM to calculate on a single path, our TSM method
  leverages the inversion process of DDIM to generate two paths from the same
  starting point for calculation. Since both paths start from the same starting
  point, TSM can reduce the accumulated error compared to ISM, thus alleviating
  the problem of pseudo ground truth inconsistency. TSM enhances the stability
  and consistency of the model's generated paths during the distillation process.
  We demonstrate this experimentally and further show that ISM is a special case
  of TSM. Furthermore, to optimize the current multi-stage optimization process
  from high-resolution text to 3D generation, we adopt Stable Diffusion XL for
  guidance. In response to the issues of abnormal replication and splitting
  caused by unstable gradients during the 3D Gaussian splatting process when
  using Stable Diffusion XL, we propose a pixel-by-pixel gradient clipping
  method. Extensive experiments show that our model significantly surpasses the
  state-of-the-art models in terms of visual quality and performance. Code:
  https://github.com/xingy038/Dreamer-XL.
paper_ai_keywords: Trajectory Score Matching (TSM) Interval Score Matching (ISM) Denoising Diffusion Implicit Models (DDIM) Stable Diffusion XL Gaussian splatting pixel-by-pixel gradient clipping