Dual Caption Preference Optimization for Diffusion Models
Recent advancements in human preference optimization, originally developed
for Large Language Models (LLMs), have shown significant potential in improving
text-to-image diffusion models. These methods aim to learn the distribution of
preferred samples while distinguishing them from less preferred ones. However,
existing preference datasets often exhibit overlap between these distributions,
leading to a conflict distribution. Additionally, we identified that input
prompts contain irrelevant information for less preferred images, limiting the
denoising network's ability to accurately predict noise in preference
optimization methods, known as the irrelevant prompt issue. To address these
challenges, we propose Dual Caption Preference Optimization (DCPO), a novel
approach that utilizes two distinct captions to mitigate irrelevant prompts. To
tackle conflict distribution, we introduce the Pick-Double Caption dataset, a
modified version of Pick-a-Pic v2 with separate captions for preferred and less
preferred images. We further propose three different strategies for generating
distinct captions: captioning, perturbation, and hybrid methods. Our
experiments show that DCPO significantly improves image quality and relevance
to prompts, outperforming Stable Diffusion (SD) 2.1, SFT_Chosen, Diffusion-DPO,
and MaPO across multiple metrics, including Pickscore, HPSv2.1, GenEval,
CLIPscore, and ImageReward, fine-tuned on SD 2.1 as the backbone.
67aac3a9ef5570c0c9047095
Amir Saeidi
extracted_confirmed
2025-02-11T03:31:48.492Z
640f6299ef5c6dcac8b1df52
/avatars/022f21183abc8a8b5ce1b198d3ba96dc.svg
Amir
user
sahsaeedi
67aac3a9ef5570c0c9047096
Yiran Luo
claimed_verified
2025-03-16T21:19:35.449Z
62917f8ba391a907d5e524a4
https://cdn-avatars.huggingface.co/v1/production/uploads/62917f8ba391a907d5e524a4/Ts2V9JtcJk-0z1pBuxDur.jpeg
Yiran Lawrence Luo
user
fpsluozi
67aac3a9ef5570c0c9047097
Agneet Chatterjee
claimed_verified
2025-02-13T08:25:34.589Z
6320c537a023aad6a7680c8b
/avatars/057dc492b8f756b83f12ced0b74fae65.svg
Agneet Chatterjee
user
agneet
67aac3a9ef5570c0c9047098
Shamanthak Hegde
claimed_verified
2025-04-03T08:29:43.407Z
66fa328b01bae9d4a42c3148
/avatars/9bce0dd6f2edda3277364f1f50c4e5ca.svg
Shamanthak Hegde
user
shamanthakhegde
67aac3a9ef5570c0c9047099
Bimsara Pathiraja
67aac3a9ef5570c0c904709a
Yezhou Yang
67aac3a9ef5570c0c904709b
Chitta Baral
Dual Caption Preference Optimization for Diffusion Models
Recent advancements in human preference optimization, originally developed
for Large Language Models (LLMs), have shown significant potential in improving
text-to-image diffusion models. These methods aim to learn the distribution of
preferred samples while distinguishing them from less preferred ones. However,
existing preference datasets often exhibit overlap between these distributions,
leading to a conflict distribution. Additionally, we identified that input
prompts contain irrelevant information for less preferred images, limiting the
denoising network's ability to accurately predict noise in preference
optimization methods, known as the irrelevant prompt issue. To address these
challenges, we propose Dual Caption Preference Optimization (DCPO), a novel
approach that utilizes two distinct captions to mitigate irrelevant prompts. To
tackle conflict distribution, we introduce the Pick-Double Caption dataset, a
modified version of Pick-a-Pic v2 with separate captions for preferred and less
preferred images. We further propose three different strategies for generating
distinct captions: captioning, perturbation, and hybrid methods. Our
experiments show that DCPO significantly improves image quality and relevance
to prompts, outperforming Stable Diffusion (SD) 2.1, SFT_Chosen, Diffusion-DPO,
and MaPO across multiple metrics, including Pickscore, HPSv2.1, GenEval,
CLIPscore, and ImageReward, fine-tuned on SD 2.1 as the backbone.
A novel preference optimization method using dual captions improves text-to-image diffusion models by addressing conflicting distributions and irrelevant prompts.
human preference optimization Large Language Models (LLMs) text-to-image diffusion models denoising network irrelevant prompt issue Dual Caption Preference Optimization (DCPO) Pick-Double Caption dataset Pickscore HPSv2.1 GenEval CLIPscore ImageReward