Use `float8_weight_only` and `float8_dynamic_activation_float8_weight` to quantize the text encoder and transformer model.  
The default quantization method is per tensor quantization, but if your GPU supports row-wise quantization, you can also try it for better accuracy.  
Install [torchao](https://github.com/pytorch/ao/tree/main) with the command below.  
```bash
pip3 install -U torch torchao
```