title: Dual Caption Preference Optimization for Diffusion Models
summary: Recent advancements in human preference optimization, originally developed
  for Large Language Models (LLMs), have shown significant potential in improving
  text-to-image diffusion models. These methods aim to learn the distribution of
  preferred samples while distinguishing them from less preferred ones. However,
  existing preference datasets often exhibit overlap between these distributions,
  leading to a conflict distribution. Additionally, we identified that input
  prompts contain irrelevant information for less preferred images, limiting the
  denoising network's ability to accurately predict noise in preference
  optimization methods, known as the irrelevant prompt issue. To address these
  challenges, we propose Dual Caption Preference Optimization (DCPO), a novel
  approach that utilizes two distinct captions to mitigate irrelevant prompts. To
  tackle conflict distribution, we introduce the Pick-Double Caption dataset, a
  modified version of Pick-a-Pic v2 with separate captions for preferred and less
  preferred images. We further propose three different strategies for generating
  distinct captions: captioning, perturbation, and hybrid methods. Our
  experiments show that DCPO significantly improves image quality and relevance
  to prompts, outperforming Stable Diffusion (SD) 2.1, SFT_Chosen, Diffusion-DPO,
  and MaPO across multiple metrics, including Pickscore, HPSv2.1, GenEval,
  CLIPscore, and ImageReward, fine-tuned on SD 2.1 as the backbone.
paper_authors._id: 67aac3a9ef5570c0c9047095
paper_authors.name: Amir Saeidi
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-02-11T03:31:48.492Z
paper_authors._id: 640f6299ef5c6dcac8b1df52
paper_authors.avatarUrl: /avatars/022f21183abc8a8b5ce1b198d3ba96dc.svg
paper_authors.fullname: Amir
paper_authors.type: user
paper_authors.user: sahsaeedi
paper_authors._id: 67aac3a9ef5570c0c9047096
paper_authors.name: Yiran Luo
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-16T21:19:35.449Z
paper_authors._id: 62917f8ba391a907d5e524a4
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62917f8ba391a907d5e524a4/Ts2V9JtcJk-0z1pBuxDur.jpeg
paper_authors.fullname: Yiran Lawrence Luo
paper_authors.type: user
paper_authors.user: fpsluozi
paper_authors._id: 67aac3a9ef5570c0c9047097
paper_authors.name: Agneet Chatterjee
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-13T08:25:34.589Z
paper_authors._id: 6320c537a023aad6a7680c8b
paper_authors.avatarUrl: /avatars/057dc492b8f756b83f12ced0b74fae65.svg
paper_authors.fullname: Agneet Chatterjee
paper_authors.type: user
paper_authors.user: agneet
paper_authors._id: 67aac3a9ef5570c0c9047098
paper_authors.name: Shamanthak Hegde
paper_authors._id: 67aac3a9ef5570c0c9047099
paper_authors.name: Bimsara Pathiraja
paper_authors._id: 67aac3a9ef5570c0c904709a
paper_authors.name: Yezhou Yang
paper_authors._id: 67aac3a9ef5570c0c904709b
paper_authors.name: Chitta Baral
paper_title: Dual Caption Preference Optimization for Diffusion Models
paper_summary: Recent advancements in human preference optimization, originally developed
  for Large Language Models (LLMs), have shown significant potential in improving
  text-to-image diffusion models. These methods aim to learn the distribution of
  preferred samples while distinguishing them from less preferred ones. However,
  existing preference datasets often exhibit overlap between these distributions,
  leading to a conflict distribution. Additionally, we identified that input
  prompts contain irrelevant information for less preferred images, limiting the
  denoising network's ability to accurately predict noise in preference
  optimization methods, known as the irrelevant prompt issue. To address these
  challenges, we propose Dual Caption Preference Optimization (DCPO), a novel
  approach that utilizes two distinct captions to mitigate irrelevant prompts. To
  tackle conflict distribution, we introduce the Pick-Double Caption dataset, a
  modified version of Pick-a-Pic v2 with separate captions for preferred and less
  preferred images. We further propose three different strategies for generating
  distinct captions: captioning, perturbation, and hybrid methods. Our
  experiments show that DCPO significantly improves image quality and relevance
  to prompts, outperforming Stable Diffusion (SD) 2.1, SFT_Chosen, Diffusion-DPO,
  and MaPO across multiple metrics, including Pickscore, HPSv2.1, GenEval,
  CLIPscore, and ImageReward, fine-tuned on SD 2.1 as the backbone.
paper_ai_keywords: text-to-image diffusion models preference datasets conflict distribution denoising network irrelevant prompt issue Dual Caption Preference Optimization (DCPO) Pick-Double Caption dataset Pick-a-Pic v2 captioning perturbation hybrid methods Pickscore HPSv2.1 GenEval CLIPscore ImageReward Stable Diffusion (SD) 2.1 SFT_Chosen Diffusion-DPO MaPO