We recommend [xFormers](https://github.com/facebookresearch/xformers) for both inference and training. In our tests, the optimizations performed in the attention blocks allow for both faster speed and reduced memory consumption.  
Install xFormers from `pip`:  
```bash
pip install xformers
```  
<Tip>