Update README.md to include a quantization code snippet

#7
by sayakpaul HF Staff - opened
Files changed (1) hide show
  1. README.md +26 -0
README.md CHANGED
@@ -90,6 +90,32 @@ image = pipe(
90
  image.save("example.png")
91
  ```
92
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
93
  ## Show Cases
94
 
95
  One of its standout capabilities is high-fidelity text rendering across diverse images. Whether it’s alphabetic languages like English or logographic scripts like Chinese, Qwen-Image preserves typographic details, layout coherence, and contextual harmony with stunning accuracy. Text isn’t just overlaid—it’s seamlessly integrated into the visual fabric.
 
90
  image.save("example.png")
91
  ```
92
 
93
+ ## Running with quantization
94
+
95
+ If you're working with a resource-constained environment, consider applying quantization. Below, we provide a snippet of using the NF4 quantization scheme through `bitsandbytes`.
96
+
97
+ ```py
98
+ # make sure bitsandbytes is installed: `pip install -U bitsandbytes
99
+
100
+ from diffusers import DiffusionPipeline, PipelineQuantizationConfig
101
+ import torch
102
+
103
+ quant_config = PipelineQuantizationConfig(
104
+ quant_backend="bitsandbytes_4bit",
105
+ quant_kwargs={"load_in_4bit": True, "bnb_4bit_quant_type": "nf4", "bnb_4bit_compute_dtype": torch.bfloat16},
106
+ components_to_quantize=["transformer", "text_encoder"],
107
+ )
108
+ pipe = DiffusionPipeline.from_pretrained(
109
+ "Qwen/Qwen-Image",
110
+ quantization_config=quant_config,
111
+ torch_dtype=torch.bfloat16
112
+ ).to("cuda")
113
+
114
+ prompt = "A cat holding a sign that says hello world"
115
+ image = pipe(prompt, num_inference_steps=4, guidance_scale=0.0).images[0]
116
+ image.save("qwenimage_nf4.png")
117
+ ```
118
+
119
  ## Show Cases
120
 
121
  One of its standout capabilities is high-fidelity text rendering across diverse images. Whether it’s alphabetic languages like English or logographic scripts like Chinese, Qwen-Image preserves typographic details, layout coherence, and contextual harmony with stunning accuracy. Text isn’t just overlaid—it’s seamlessly integrated into the visual fabric.