title: Diffusion-Sharpening: Fine-tuning Diffusion Models with Denoising
    Trajectory Sharpening
summary: We propose Diffusion-Sharpening, a fine-tuning approach that enhances
  downstream alignment by optimizing sampling trajectories. Existing RL-based
  fine-tuning methods focus on single training timesteps and neglect
  trajectory-level alignment, while recent sampling trajectory optimization
  methods incur significant inference NFE costs. Diffusion-Sharpening overcomes
  this by using a path integral framework to select optimal trajectories during
  training, leveraging reward feedback, and amortizing inference costs. Our
  method demonstrates superior training efficiency with faster convergence, and
  best inference efficiency without requiring additional NFEs. Extensive
  experiments show that Diffusion-Sharpening outperforms RL-based fine-tuning
  methods (e.g., Diffusion-DPO) and sampling trajectory optimization methods
  (e.g., Inference Scaling) across diverse metrics including text alignment,
  compositional capabilities, and human preferences, offering a scalable and
  efficient solution for future diffusion model fine-tuning. Code:
  https://github.com/Gen-Verse/Diffusion-Sharpening
paper_authors._id: 67b40ce4d3c5f50aa9b71df5
paper_authors.name: Ye Tian
paper_authors._id: 67b40ce4d3c5f50aa9b71df6
paper_authors.name: Ling Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-20T09:37:40.151Z
paper_authors._id: 64fde4e252e82dd432b74ce9
paper_authors.avatarUrl: /avatars/061a69d858b86d1600be916122cae7fc.svg
paper_authors.fullname: Ling Yang
paper_authors.type: user
paper_authors.user: Lingaaaaaaa
paper_authors._id: 67b40ce4d3c5f50aa9b71df7
paper_authors.name: Xinchen Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-18T09:31:29.697Z
paper_authors._id: 653e5d31ffd60206c8b64bb5
paper_authors.avatarUrl: /avatars/5076795722ec1f9e031654f301d30e8f.svg
paper_authors.fullname: Xinchen Zhang
paper_authors.type: user
paper_authors.user: comin
paper_authors._id: 67b40ce4d3c5f50aa9b71df8
paper_authors.name: Yunhai Tong
paper_authors._id: 67b40ce4d3c5f50aa9b71df9
paper_authors.name: Mengdi Wang
paper_authors._id: 67b40ce4d3c5f50aa9b71dfa
paper_authors.name: Bin Cui
paper_title: Diffusion-Sharpening: Fine-tuning Diffusion Models with Denoising
    Trajectory Sharpening
paper_summary: We propose Diffusion-Sharpening, a fine-tuning approach that enhances
  downstream alignment by optimizing sampling trajectories. Existing RL-based
  fine-tuning methods focus on single training timesteps and neglect
  trajectory-level alignment, while recent sampling trajectory optimization
  methods incur significant inference NFE costs. Diffusion-Sharpening overcomes
  this by using a path integral framework to select optimal trajectories during
  training, leveraging reward feedback, and amortizing inference costs. Our
  method demonstrates superior training efficiency with faster convergence, and
  best inference efficiency without requiring additional NFEs. Extensive
  experiments show that Diffusion-Sharpening outperforms RL-based fine-tuning
  methods (e.g., Diffusion-DPO) and sampling trajectory optimization methods
  (e.g., Inference Scaling) across diverse metrics including text alignment,
  compositional capabilities, and human preferences, offering a scalable and
  efficient solution for future diffusion model fine-tuning. Code:
  https://github.com/Gen-Verse/Diffusion-Sharpening
paper_ai_keywords: Diffusion-Sharpening fine-tuning approach downstream alignment sampling trajectories RL-based fine-tuning methods trajectory-level alignment path integral framework optimal trajectories reward feedback amortizing inference costs training efficiency convergence inference efficiency NFE costs Diffusion-DPO Inference Scaling text alignment compositional capabilities human preferences scalable solution