Is there or will there be support for xformers?
### Feature request

xformers, (i couldn't find anything online or in the docs, but i suspect its very likely I'm just missing something)

### Motivation

speed and memory improvement

### Your contribution

I am unsure, but willing to help.
cc @younesbelkada <|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.