Update README.md
Browse files
README.md
CHANGED
|
@@ -26,7 +26,7 @@ Key benefits:
|
|
| 26 |
|
| 27 |
### 🔧 How to Use
|
| 28 |
|
| 29 |
-
1. Install the DFloat11
|
| 30 |
|
| 31 |
```bash
|
| 32 |
pip install dfloat11[cuda12]
|
|
@@ -34,31 +34,56 @@ Key benefits:
|
|
| 34 |
# pip install dfloat11[cuda11]
|
| 35 |
```
|
| 36 |
|
| 37 |
-
2.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 38 |
```python
|
| 39 |
import torch
|
| 40 |
-
from diffusers import FluxPipeline
|
| 41 |
from dfloat11 import DFloat11Model
|
| 42 |
-
|
| 43 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 44 |
pipe.enable_model_cpu_offload()
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
prompt = "A futuristic cityscape at sunset, with flying cars, neon lights, and reflective water canals"
|
| 49 |
image = pipe(
|
| 50 |
prompt,
|
| 51 |
-
width=
|
| 52 |
-
height=
|
| 53 |
guidance_scale=3.5,
|
| 54 |
num_inference_steps=50,
|
| 55 |
max_sequence_length=512,
|
| 56 |
generator=torch.Generator(device="cuda").manual_seed(0)
|
| 57 |
).images[0]
|
| 58 |
-
|
| 59 |
image.save("image.png")
|
| 60 |
```
|
| 61 |
|
|
|
|
|
|
|
| 62 |
### 📄 Learn More
|
| 63 |
|
| 64 |
* **Paper**: [70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float](https://arxiv.org/abs/2504.11651)
|
|
|
|
| 26 |
|
| 27 |
### 🔧 How to Use
|
| 28 |
|
| 29 |
+
1. Install or upgrade the DFloat11 package *(installs the CUDA kernel automatically; requires a CUDA-compatible GPU and PyTorch installed)*:
|
| 30 |
|
| 31 |
```bash
|
| 32 |
pip install dfloat11[cuda12]
|
|
|
|
| 34 |
# pip install dfloat11[cuda11]
|
| 35 |
```
|
| 36 |
|
| 37 |
+
2. Install or upgrade the diffusers package:
|
| 38 |
+
|
| 39 |
+
```bash
|
| 40 |
+
pip install -U diffusers
|
| 41 |
+
```
|
| 42 |
+
|
| 43 |
+
3. Save the following code as a Python file `flux1.py`:
|
| 44 |
```python
|
| 45 |
import torch
|
| 46 |
+
from diffusers import FluxPipeline, FluxTransformer2DModel
|
| 47 |
from dfloat11 import DFloat11Model
|
| 48 |
+
from transformers.modeling_utils import no_init_weights
|
| 49 |
+
|
| 50 |
+
with no_init_weights():
|
| 51 |
+
transformer = FluxTransformer2DModel.from_config(
|
| 52 |
+
FluxTransformer2DModel.load_config(
|
| 53 |
+
"black-forest-labs/FLUX.1-dev", subfolder="transformer"
|
| 54 |
+
)
|
| 55 |
+
).to(torch.bfloat16)
|
| 56 |
+
|
| 57 |
+
pipe = FluxPipeline.from_pretrained(
|
| 58 |
+
"black-forest-labs/FLUX.1-dev",
|
| 59 |
+
transformer=transformer,
|
| 60 |
+
torch_dtype=torch.bfloat16
|
| 61 |
+
)
|
| 62 |
+
|
| 63 |
+
DFloat11Model.from_pretrained(
|
| 64 |
+
'DFloat11/FLUX.1-dev-DF11',
|
| 65 |
+
device='cpu',
|
| 66 |
+
bfloat16_model=pipe.transformer,
|
| 67 |
+
)
|
| 68 |
+
|
| 69 |
pipe.enable_model_cpu_offload()
|
| 70 |
+
|
| 71 |
+
prompt = "A scenic landscape with mountains, a river, and a clear sky."
|
|
|
|
|
|
|
| 72 |
image = pipe(
|
| 73 |
prompt,
|
| 74 |
+
width=1024,
|
| 75 |
+
height=1024,
|
| 76 |
guidance_scale=3.5,
|
| 77 |
num_inference_steps=50,
|
| 78 |
max_sequence_length=512,
|
| 79 |
generator=torch.Generator(device="cuda").manual_seed(0)
|
| 80 |
).images[0]
|
| 81 |
+
|
| 82 |
image.save("image.png")
|
| 83 |
```
|
| 84 |
|
| 85 |
+
4. Run `python flux1.py` in your terminal.
|
| 86 |
+
|
| 87 |
### 📄 Learn More
|
| 88 |
|
| 89 |
* **Paper**: [70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float](https://arxiv.org/abs/2504.11651)
|