Update README.md
Browse files
README.md
CHANGED
|
@@ -1,5 +1,107 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: other
|
| 3 |
-
license_name: tencent-hunyuan-community
|
| 4 |
-
license_link: https://huggingface.co/Tencent-Hunyuan/HunyuanDiT/blob/main/LICENSE.txt
|
| 5 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
license_name: tencent-hunyuan-community
|
| 4 |
+
license_link: https://huggingface.co/Tencent-Hunyuan/HunyuanDiT/blob/main/LICENSE.txt
|
| 5 |
+
language:
|
| 6 |
+
- en
|
| 7 |
+
---
|
| 8 |
+
<!-- ## **HunyuanDiT** -->
|
| 9 |
+
|
| 10 |
+
<p align="center">
|
| 11 |
+
<img src="https://raw.githubusercontent.com/Tencent/HunyuanDiT/main/asset/logo.png" height=100>
|
| 12 |
+
</p>
|
| 13 |
+
|
| 14 |
+
# Hunyuan-DiT : A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding
|
| 15 |
+
# 混元-DiT: 具有细粒度中文理解的多分辨率Diffusion Transformer
|
| 16 |
+
|
| 17 |
+
[[Arxiv]](https://arxiv.org/abs/2405.08748) [[project page]](https://dit.hunyuan.tencent.com/) [[github]](https://github.com/Tencent/HunyuanDiT)
|
| 18 |
+
|
| 19 |
+
This repo contains the distilled Hunyuan-DiT in 🤗 [Diffusers](https://github.com/huggingface/diffusers) format.
|
| 20 |
+
|
| 21 |
+
It supports 25-step text-to-image generation.
|
| 22 |
+
|
| 23 |
+
## Dependency
|
| 24 |
+
Please install PyTorch first, following the instruction in [https://pytorch.org](https://pytorch.org)
|
| 25 |
+
|
| 26 |
+
Install the latest version of transformers with `pip`:
|
| 27 |
+
```
|
| 28 |
+
pip install --upgrade transformers
|
| 29 |
+
```
|
| 30 |
+
|
| 31 |
+
Then install the latest github version of 🤗 Diffusers with `pip`:
|
| 32 |
+
```
|
| 33 |
+
pip install git+https://github.com/huggingface/diffusers.git
|
| 34 |
+
```
|
| 35 |
+
|
| 36 |
+
## Example Usage with 🤗 Diffusers
|
| 37 |
+
```py
|
| 38 |
+
import torch
|
| 39 |
+
from diffusers import HunyuanDiTPipeline
|
| 40 |
+
|
| 41 |
+
pipe = HunyuanDiTPipeline.from_pretrained("Tencent-Hunyuan/HunyuanDiT-Diffusers-Distilled", torch_dtype=torch.float16)
|
| 42 |
+
pipe.to("cuda")
|
| 43 |
+
|
| 44 |
+
# You may also use English prompt as HunyuanDiT supports both English and Chinese
|
| 45 |
+
# prompt = "An astronaut riding a horse"
|
| 46 |
+
prompt = "一个宇航员在骑马"
|
| 47 |
+
image = pipe(prompt).images[0]
|
| 48 |
+
```
|
| 49 |
+
|
| 50 |
+

|
| 51 |
+
|
| 52 |
+
## 📈 Comparisons
|
| 53 |
+
In order to comprehensively compare the generation capabilities of HunyuanDiT and other models, we constructed a 4-dimensional test set, including Text-Image Consistency, Excluding AI Artifacts, Subject Clarity, Aesthetic. More than 50 professional evaluators performs the evaluation.
|
| 54 |
+
|
| 55 |
+
<p align="center">
|
| 56 |
+
<table>
|
| 57 |
+
<thead>
|
| 58 |
+
<tr>
|
| 59 |
+
<th rowspan="2">Model</th> <th rowspan="2">Open Source</th> <th>Text-Image Consistency (%)</th> <th>Excluding AI Artifacts (%)</th> <th>Subject Clarity (%)</th> <th rowspan="2">Aesthetics (%)</th> <th rowspan="2">Overall (%)</th>
|
| 60 |
+
</tr>
|
| 61 |
+
</thead>
|
| 62 |
+
<tbody>
|
| 63 |
+
<tr>
|
| 64 |
+
<td>SDXL</td> <td> ✔ </td> <td>64.3</td> <td>60.6</td> <td>91.1</td> <td>76.3</td> <td>42.7</td>
|
| 65 |
+
</tr>
|
| 66 |
+
<tr>
|
| 67 |
+
<td>PixArt-α</td> <td> ✔</td> <td>68.3</td> <td>60.9</td> <td>93.2</td> <td>77.5</td> <td>45.5</td>
|
| 68 |
+
</tr>
|
| 69 |
+
<tr>
|
| 70 |
+
<td>Playground 2.5</td> <td>✔</td> <td>71.9</td> <td>70.8</td> <td>94.9</td> <td>83.3</td> <td>54.3</td>
|
| 71 |
+
</tr>
|
| 72 |
+
|
| 73 |
+
<tr>
|
| 74 |
+
<td>SD 3</td> <td>✘</td> <td>77.1</td> <td>69.3</td> <td>94.6</td> <td>82.5</td> <td>56.7</td>
|
| 75 |
+
|
| 76 |
+
</tr>
|
| 77 |
+
<tr>
|
| 78 |
+
<td>MidJourney v6</td><td>✘</td> <td>73.5</td> <td>80.2</td> <td>93.5</td> <td>87.2</td> <td>63.3</td>
|
| 79 |
+
</tr>
|
| 80 |
+
<tr>
|
| 81 |
+
<td>DALL-E 3</td><td>✘</td> <td>83.9</td> <td>80.3</td> <td>96.5</td> <td>89.4</td> <td>71.0</td>
|
| 82 |
+
</tr>
|
| 83 |
+
<tr style="font-weight: bold; background-color: #f2f2f2;">
|
| 84 |
+
<td>Hunyuan-DiT</td><td>✔</td> <td>74.2</td> <td>74.3</td> <td>95.4</td> <td>86.6</td> <td>59.0</td>
|
| 85 |
+
</tr>
|
| 86 |
+
</tbody>
|
| 87 |
+
</table>
|
| 88 |
+
</p>
|
| 89 |
+
|
| 90 |
+
## 🎥 Visualization
|
| 91 |
+
|
| 92 |
+
* **Chinese Elements**
|
| 93 |
+
<p align="center">
|
| 94 |
+
<img src="https://raw.githubusercontent.com/Tencent/HunyuanDiT/main/asset/chinese elements understanding.png" height=220>
|
| 95 |
+
</p>
|
| 96 |
+
|
| 97 |
+
* **Long Text Input**
|
| 98 |
+
|
| 99 |
+
|
| 100 |
+
<p align="center">
|
| 101 |
+
<img src="https://raw.githubusercontent.com/Tencent/HunyuanDiT/main/asset/long text understanding.png" height=310>
|
| 102 |
+
</p>
|
| 103 |
+
|
| 104 |
+
|
| 105 |
+
## 🔥🔥🔥 Tencent Hunyuan Bot
|
| 106 |
+
|
| 107 |
+
Welcome to [Tencent Hunyuan Bot](https://hunyuan.tencent.com/bot/chat), where you can explore our innovative products in multi-round conversation!
|