Update README.md
#50
by
Yanzuo
- opened
README.md
CHANGED
@@ -18,8 +18,9 @@ Project Page: https://hyper-sd.github.io/
|
|
18 |
|
19 |
## Newsπ₯π₯π₯
|
20 |
|
21 |
-
*
|
22 |
-
*
|
|
|
23 |
* Apr.28, 2024. ComfyUI workflows on 1-Step Unified LoRA π₯° with TCDScheduler to inference on different steps are [released](https://huggingface.co/ByteDance/Hyper-SD/tree/main/comfyui)! Remember to install βοΈ [ComfyUI-TCD](https://github.com/JettHu/ComfyUI-TCD) in your `ComfyUI/custom_nodes` folder!!! You're encouraged to adjust the eta parameter to get better results π!
|
24 |
* Apr.26, 2024. Thanks to @[Pete](https://huggingface.co/pngwn) for contributing to our [scribble demo](https://huggingface.co/spaces/ByteDance/Hyper-SD15-Scribble) with larger canvas right now π.
|
25 |
* Apr.24, 2024. The ComfyUI [workflow](https://huggingface.co/ByteDance/Hyper-SD/blob/main/comfyui/Hyper-SDXL-1step-Unet-workflow.json) and [checkpoint](https://huggingface.co/ByteDance/Hyper-SD/blob/main/Hyper-SDXL-1step-Unet-Comfyui.fp16.safetensors) on 1-Step SDXL UNet β¨ is also available! Don't forget βοΈ to install the custom [scheduler](https://huggingface.co/ByteDance/Hyper-SD/tree/main/comfyui/ComfyUI-HyperSDXL1StepUnetScheduler) in your `ComfyUI/custom_nodes` folder!!!
|
@@ -36,15 +37,36 @@ Hyper-SDXL One-step Text-to-Image demo host on [π€ T2I](https://huggingface.co
|
|
36 |
## Introduction
|
37 |
|
38 |
Hyper-SD is one of the new State-of-the-Art diffusion model acceleration techniques.
|
39 |
-
In this repository, we release the models distilled from [SDXL Base 1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0) and [Stable-Diffusion v1-5](https://huggingface.co/runwayml/stable-diffusion-v1-5)γ
|
40 |
|
41 |
## Checkpoints
|
42 |
|
|
|
43 |
* `Hyper-SDXL-Nstep-lora.safetensors`: Lora checkpoint, for SDXL-related models.
|
44 |
* `Hyper-SD15-Nstep-lora.safetensors`: Lora checkpoint, for SD1.5-related models.
|
45 |
* `Hyper-SDXL-1step-unet.safetensors`: Unet checkpoint distilled from SDXL-Base.
|
46 |
|
47 |
## Text-to-Image Usage
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
48 |
### SDXL-related models
|
49 |
#### 2-Steps, 4-Steps, 8-steps LoRA
|
50 |
Take the 2-steps LoRA as an example, you can also use other LoRAs for the corresponding inference steps setting.
|
|
|
18 |
|
19 |
## Newsπ₯π₯π₯
|
20 |
|
21 |
+
* Aug.19, 2024. π₯π₯π₯ **SD3-related CFG LoRAs** are available now! We recommend setting guidance scale to 3.0/5.0/7.0 at 4/8/16-steps. Don't forget to fuse lora with a relatively small scale (e.g. 0.125 that is adaptive with training) before inference with diffusers. Note that 8-steps and 16-steps LoRA can also inference on a little bit smaller steps like 6-steps and 12-steps, respectively. Hope to hear your feedback, FLUX-related models will be coming next week. π₯π₯π₯
|
22 |
+
* May.13, 2024. The 12-Steps CFG-Preserved [Hyper-SDXL-12steps-CFG-LoRA](https://huggingface.co/ByteDance/Hyper-SD/blob/main/Hyper-SDXL-12steps-CFG-lora.safetensors) and [Hyper-SD15-12steps-CFG-LoRA](https://huggingface.co/ByteDance/Hyper-SD/blob/main/Hyper-SD15-12steps-CFG-lora.safetensors) is also available now(support 5~8 guidance scales), this could be more practical with better trade-off between performance and speed. Enjoy!
|
23 |
+
* Apr.30, 2024. Our 8-Steps CFG-Preserved [Hyper-SDXL-8steps-CFG-LoRA](https://huggingface.co/ByteDance/Hyper-SD/blob/main/Hyper-SDXL-8steps-CFG-lora.safetensors) and [Hyper-SD15-8steps-CFG-LoRA](https://huggingface.co/ByteDance/Hyper-SD/blob/main/Hyper-SD15-8steps-CFG-lora.safetensors) is available now(support 5~8 guidance scales), we strongly recommend making the 8-step CFGLora a standard configuration for all SDXL and SD15 models!!!
|
24 |
* Apr.28, 2024. ComfyUI workflows on 1-Step Unified LoRA π₯° with TCDScheduler to inference on different steps are [released](https://huggingface.co/ByteDance/Hyper-SD/tree/main/comfyui)! Remember to install βοΈ [ComfyUI-TCD](https://github.com/JettHu/ComfyUI-TCD) in your `ComfyUI/custom_nodes` folder!!! You're encouraged to adjust the eta parameter to get better results π!
|
25 |
* Apr.26, 2024. Thanks to @[Pete](https://huggingface.co/pngwn) for contributing to our [scribble demo](https://huggingface.co/spaces/ByteDance/Hyper-SD15-Scribble) with larger canvas right now π.
|
26 |
* Apr.24, 2024. The ComfyUI [workflow](https://huggingface.co/ByteDance/Hyper-SD/blob/main/comfyui/Hyper-SDXL-1step-Unet-workflow.json) and [checkpoint](https://huggingface.co/ByteDance/Hyper-SD/blob/main/Hyper-SDXL-1step-Unet-Comfyui.fp16.safetensors) on 1-Step SDXL UNet β¨ is also available! Don't forget βοΈ to install the custom [scheduler](https://huggingface.co/ByteDance/Hyper-SD/tree/main/comfyui/ComfyUI-HyperSDXL1StepUnetScheduler) in your `ComfyUI/custom_nodes` folder!!!
|
|
|
37 |
## Introduction
|
38 |
|
39 |
Hyper-SD is one of the new State-of-the-Art diffusion model acceleration techniques.
|
40 |
+
In this repository, we release the models distilled from [SD3-Medium](https://huggingface.co/stabilityai/stable-diffusion-3-medium-diffusers), [SDXL Base 1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0) and [Stable-Diffusion v1-5](https://huggingface.co/runwayml/stable-diffusion-v1-5)γ
|
41 |
|
42 |
## Checkpoints
|
43 |
|
44 |
+
* `Hyper-SD3-Nsteps-CFG-lora.safetensors`: Lora checkpoint, for SD3-related models.
|
45 |
* `Hyper-SDXL-Nstep-lora.safetensors`: Lora checkpoint, for SDXL-related models.
|
46 |
* `Hyper-SD15-Nstep-lora.safetensors`: Lora checkpoint, for SD1.5-related models.
|
47 |
* `Hyper-SDXL-1step-unet.safetensors`: Unet checkpoint distilled from SDXL-Base.
|
48 |
|
49 |
## Text-to-Image Usage
|
50 |
+
|
51 |
+
### SD3-related models
|
52 |
+
```python
|
53 |
+
import torch
|
54 |
+
from diffusers import StableDiffusion3Pipeline
|
55 |
+
from huggingface_hub import hf_hub_download
|
56 |
+
base_model_id = "stabilityai/stable-diffusion-3-medium-diffusers"
|
57 |
+
repo_name = "ByteDance/Hyper-SD"
|
58 |
+
# Take 8-steps lora as an example
|
59 |
+
ckpt_name = "Hyper-SD3-8steps-CFG-lora.safetensors"
|
60 |
+
# Load model, please fill in your access tokens since SD3 repo is a gated model.
|
61 |
+
pipe = StableDiffusion3Pipeline.from_pretrained(base_model_id, token="xxx")
|
62 |
+
pipe.load_lora_weights(hf_hub_download(repo_name, ckpt_name))
|
63 |
+
pipe.fuse_lora(lora_scale=0.125)
|
64 |
+
pipe.to("cuda", dtype=torch.float16)
|
65 |
+
image=pipe(prompt="a photo of a cat", num_inference_steps=8, guidance_scale=5.0).images[0]
|
66 |
+
image.save("output.png")
|
67 |
+
|
68 |
+
```
|
69 |
+
|
70 |
### SDXL-related models
|
71 |
#### 2-Steps, 4-Steps, 8-steps LoRA
|
72 |
Take the 2-steps LoRA as an example, you can also use other LoRAs for the corresponding inference steps setting.
|