title: Aligning Generative Denoising with Discriminative Objectives Unleashes
    Diffusion for Visual Perception
summary: With the success of image generation, generative diffusion models are
  increasingly adopted for discriminative tasks, as pixel generation provides a
  unified perception interface. However, directly repurposing the generative
  denoising process for discriminative objectives reveals critical gaps rarely
  addressed previously. Generative models tolerate intermediate sampling errors
  if the final distribution remains plausible, but discriminative tasks require
  rigorous accuracy throughout, as evidenced in challenging multi-modal tasks
  like referring image segmentation. Motivated by this gap, we analyze and
  enhance alignment between generative diffusion processes and perception tasks,
  focusing on how perception quality evolves during denoising. We find: (1)
  earlier denoising steps contribute disproportionately to perception quality,
  prompting us to propose tailored learning objectives reflecting varying
  timestep contributions; (2) later denoising steps show unexpected perception
  degradation, highlighting sensitivity to training-denoising distribution
  shifts, addressed by our diffusion-tailored data augmentation; and (3)
  generative processes uniquely enable interactivity, serving as controllable
  user interfaces adaptable to correctional prompts in multi-round interactions.
  Our insights significantly improve diffusion-based perception models without
  architectural changes, achieving state-of-the-art performance on depth
  estimation, referring image segmentation, and generalist perception tasks. Code
  available at https://github.com/ziqipang/ADDP.
paper_authors._id: 67ffcf5874193e65a7831a17
paper_authors.name: Ziqi Pang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-17T11:16:13.569Z
paper_authors._id: 642a33ea5673845d9854f458
paper_authors.avatarUrl: /avatars/ea8ed29a0b4ad629a2ba6e7d26cbd923.svg
paper_authors.fullname: Ziqi Pang
paper_authors.type: user
paper_authors.user: ziqipang
paper_authors._id: 67ffcf5874193e65a7831a18
paper_authors.name: Xin Xu
paper_authors._id: 67ffcf5874193e65a7831a19
paper_authors.name: Yu-Xiong Wang
paper_title: Aligning Generative Denoising with Discriminative Objectives Unleashes
    Diffusion for Visual Perception
paper_summary: With the success of image generation, generative diffusion models are
  increasingly adopted for discriminative tasks, as pixel generation provides a
  unified perception interface. However, directly repurposing the generative
  denoising process for discriminative objectives reveals critical gaps rarely
  addressed previously. Generative models tolerate intermediate sampling errors
  if the final distribution remains plausible, but discriminative tasks require
  rigorous accuracy throughout, as evidenced in challenging multi-modal tasks
  like referring image segmentation. Motivated by this gap, we analyze and
  enhance alignment between generative diffusion processes and perception tasks,
  focusing on how perception quality evolves during denoising. We find: (1)
  earlier denoising steps contribute disproportionately to perception quality,
  prompting us to propose tailored learning objectives reflecting varying
  timestep contributions; (2) later denoising steps show unexpected perception
  degradation, highlighting sensitivity to training-denoising distribution
  shifts, addressed by our diffusion-tailored data augmentation; and (3)
  generative processes uniquely enable interactivity, serving as controllable
  user interfaces adaptable to correctional prompts in multi-round interactions.
  Our insights significantly improve diffusion-based perception models without
  architectural changes, achieving state-of-the-art performance on depth
  estimation, referring image segmentation, and generalist perception tasks. Code
  available at https://github.com/ziqipang/ADDP.
paper_ai_summary: Enhancements to generative diffusion models address gaps in discriminative tasks by focusing on perception quality during denoising, improving performance on depth estimation, referring image segmentation, and generalist perception tasks.
paper_ai_keywords: generative diffusion models discriminative tasks pixel generation generative denoising referring image segmentation perception quality denoising steps learning objectives diffusion-tailored data augmentation interactivity correctional prompts depth estimation generalist perception tasks