content.raw: It's been a while we shipped native quantization support in 
content.type: text
content.value: It's been a while we shipped native quantization support in 
content.code: diffusers
content.raw: `diffusers`
content.type: inline_code
content.raw:  🧨
content.type: text
content.value:  🧨
content.type: new_line
content.type: new_line
content.raw: We currently support 
content.type: text
content.value: We currently support 
content.code: bistandbytes
content.raw: `bistandbytes`
content.type: inline_code
content.raw:  as the official backend but using others like 
content.type: text
content.value:  as the official backend but using others like 
content.code: torchao
content.raw: `torchao`
content.type: inline_code
content.raw:  is already very simple. 
content.type: text
content.value:  is already very simple. 
content.type: new_line
content.type: new_line
content.raw: This post is just a reminder of what's possible:
content.type: text
content.value: This post is just a reminder of what's possible:
content.type: new_line
content.type: new_line
content.raw: 1. Loading a model with a quantization config
content.type: text
content.value: 1. Loading a model with a quantization config
content.type: new_line
content.raw: 2. Saving a model with quantization config
content.type: text
content.value: 2. Saving a model with quantization config
content.type: new_line
content.raw: 3. Loading a pre-quantized model
content.type: text
content.value: 3. Loading a pre-quantized model
content.type: new_line
content.raw: 4. 
content.type: text
content.value: 4. 
content.code: enable_model_cpu_offload()
content.raw: `enable_model_cpu_offload()`
content.type: inline_code
content.type: new_line
content.raw: 5. Training and loading LoRAs into quantized checkpoints
content.type: text
content.value: 5. Training and loading LoRAs into quantized checkpoints
content.type: new_line
content.type: new_line
content.raw: Docs:
content.type: text
content.value: Docs:
content.type: new_line
content.href: https://huggingface.co/docs/diffusers/main/en/quantization/bitsandbytes
content.raw: https://huggingface.co/docs/diffusers/main/en/quantization/bitsandbytes
content.type: link
rawContent: It's been a while we shipped native quantization support in `diffusers` 🧨
  
  We currently support `bistandbytes` as the official backend but using others like `torchao` is already very simple. 
  
  This post is just a reminder of what's possible:
  
  1. Loading a model with a quantization config
  2. Saving a model with quantization config
  3. Loading a pre-quantized model
  4. `enable_model_cpu_offload()`
  5. Training and loading LoRAs into quantized checkpoints
  
  Docs:
  https://huggingface.co/docs/diffusers/main/en/quantization/bitsandbytes
reactions.reaction: ❤️
reactions.users: s3nh YaTharThShaRma999 davidberenstein1957 John6666 clem merve lunarflu not-lain pcuenq KarthikaRajagopal
reactions.reaction: 🚀
reactions.users: John6666 clem lunarflu rwightman not-lain den0620 pcuenq OmbelineM ai-everyday
commentators._id: 61caeda441f9432649f03ab6
commentators.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61caeda441f9432649f03ab6/IazJBCi7Cr34DgZXZeI4k.jpeg
commentators.fullname: s3nh
commentators.name: s3nh
commentators.type: user