Before you begin, make sure you have Pytorch 2.5+ and TorchAO installed.  
```bash
pip install -U torch torchao
```  
Quantize a model by passing [`TorchAoConfig`] to [`~ModelMixin.from_pretrained`] (you can also load pre-quantized models). This works for any model in any modality, as long as it supports loading with [Accelerate](https://hf.co/docs/accelerate/index) and contains `torch.nn.Linear` layers.  
The example below only quantizes the weights to int8.  
```python
import torch