<h1 id="heading-752">What is DPO?</h1><p>DPO is <strong>D</strong>irect <strong>P</strong>reference <strong>O</strong>ptimization, the name given to the process whereby a diffusion model is finetuned based on human-chosen images. Meihua Dang et. al. have trained Stable Diffusion 1.5 and Stable Diffusion XL using this method and the Pick-a-Pic v2 Dataset, which can be found at <a target="_blank" rel="ugc" href="https://huggingface.co/datasets/yuvalkirstain/pickapic_v2">https://huggingface.co/datasets/yuvalkirstain/pickapic_v2</a>, and wrote a paper about it at <a rel="ugc" href="https://huggingface.co/papers/2311.12908">https://huggingface.co/papers/2311.12908</a>.</p><h1 id="heading-753">What does it Do?</h1><p>The trained DPO models have been observed to produce higher quality images than their untuned counterparts, with a <strong>significant emphasis</strong> on the adherence of the model to your prompt. These LoRA can bring that prompt adherence to other fine-tuned Stable Diffusion models.</p><p></p><h1 id="heading-1572">Who Trained This?</h1><p>These LoRA are based on the works of Meihua Dang (<a target="_blank" rel="ugc" href="https://huggingface.co/mhdang">https://huggingface.co/mhdang</a>) at</p><p><a target="_blank" rel="ugc" href="https://huggingface.co/mhdang/dpo-sdxl-text2image-v1">https://huggingface.co/mhdang/dpo-sdxl-text2image-v1</a> and <a target="_blank" rel="ugc" href="https://huggingface.co/mhdang/dpo-sd1.5-text2image-v1">https://huggingface.co/mhdang/dpo-sd1.5-text2image-v1</a>, licensed under OpenRail++.</p><p></p><p></p><h1 id="heading-1573">How were these LoRA Made?</h1><p>They were created using Kohya SS by extracting them from other OpenRail++ licensed checkpoints on CivitAI and HuggingFace.</p><p>1.5: <a target="_blank" rel="ugc" href="https://civitai.com/models/240850/sd15-direct-preference-optimization-dpo">https://civitai.com/models/240850/sd15-direct-preference-optimization-dpo</a> extracted from <a target="_blank" rel="ugc" href="https://huggingface.co/fp16-guy/Stable-Diffusion-v1-5_fp16_cleaned/blob/main/sd_1.5.safetensors">https://huggingface.co/fp16-guy/Stable-Diffusion-v1-5_fp16_cleaned/blob/main/sd_1.5.safetensors</a>.</p><p>XL: <a target="_blank" rel="ugc" href="https://civitai.com/models/238319/sd-xl-dpo-finetune-direct-preference-optimization">https://civitai.com/models/238319/sd-xl-dpo-finetune-direct-preference-optimization</a> extracted from <a target="_blank" rel="ugc" href="https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/blob/main/sd_xl_base_1.0_0.9vae.safetensors">https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/blob/main/sd_xl_base_1.0_0.9vae.safetensors</a></p><p>These are also hosted on HuggingFace at <a target="_blank" rel="ugc" href="https://huggingface.co/benjamin-paine/sd-dpo-offsets/">https://huggingface.co/benjamin-paine/sd-dpo-offsets/</a></p>
Public
SDXL 1.0
https://civitai.com/api/download/models/273996
true
https://civitai.com/api/download/models/273996
091E985B
C100EC5708
077C31ED3CD5
36692FB885E7938399DA9D2465C7E769E6E7A345B7DC61D298D8DA4403565347
08360901
C100EC5708865A649C68912CE0E541FC69CB1973FE6543310B9B81A42E15ADA3
SafeTensor
sd_xl_dpo_lora_v1.safetensors
No Pickle imports
Success
2023-12-24T22:40:50.665Z
Model
Success
UCKTJH020f-T}6t,$2WY15I9?HS$^J%MKPjX
image
https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/eaef6a90-c101-4183-a974-39f40c092f5c/width=450/4862211.jpeg
UJI=0@024:%L-pt7RkM{S4Ri%NV@~qbdIps.
image
https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/def49d32-e454-4d9b-9021-12be0ca10c32/width=450/4862214.jpeg
U6HUh5?bs:~VGaNHWqt8EMoJt7RP8wIVRjD%
image
https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/c7a9c5cb-980f-4bb8-ab12-d56a1f20f529/width=450/4903274.jpeg
SDXL - V1.0
2023-12-24T22:51:07.019Z
Public
SD 1.5
https://civitai.com/api/download/models/273993
https://civitai.com/api/download/models/273993
458D4D86
D4EC63A436
E0E84BC327A8
6031838787771EA0A3BB2752DF12F3717B3422560DC9A79E21E8CECBDF928342
76304D4F
D4EC63A436DBD20E5CAC4A3E9A1FD0C59FB2513018266252C4EE9EBEB5CE6762
SafeTensor
sd_v15_dpo_lora_v1.safetensors
No Pickle imports
Success
2023-12-24T22:25:36.159Z
Model
Success
UHEVHGv}r?}@-p-U$%ROE1$%a0D%%MjZM{R*
image
https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/7b5a8bf4-e6db-4989-896f-53173bf389bc/width=450/4862240.jpeg
UeG+8f~po}RjrqV@WVt7ayRjRkoffloLs:Rj
image
https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/25ea8efd-7ff8-4577-b7e8-2ab5b014199e/width=450/4862241.jpeg
UDI#fO_3oJ~W9GRPjsNG0*X8WBbbxtnikCV@
image
https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/1f1c3bd8-1735-42d2-b84c-daf8a66b746b/width=450/4903672.jpeg
SD 1.5 - V1.0
DPO (Direct Preference Optimization) LoRA for XL and 1.5 - OpenRail++
tool