Use memory efficient attention in CLIP
# What does this PR do?

This PR adds support for clip model to use memory efficient attention, similar to https://github.com/huggingface/diffusers/pull/532

~This is critical when you want to use stable diffusion with large batch size. Because when unet is using memory efficient attention, the bottleneck becomes the initial text encoding step.~
edit: I just tested to see what's the batch size that it becomes the bottleneck, but it seems that's not the bottleneck now. I'm not sure now. 🤔 

Anyways. With this improvement, I'm able to run stable diffusion with batch size 128 on RTX 3090, in here [cccntu/accelerated-stable-diffusion](https://github.com/cccntu/accelerated-stable-diffusion)
@patrickvonplaten

Is https://github.com/huggingface/diffusers/pull/532 for CLIP in Stable Diffusion? Is it necessary to port the implementation to `transformers` **CLIP** here to make running SD more efficient ...? @patrickvonplaten is the best one to make the decision.<|||||>Looks like this would be a great use of [custom modeling code](https://huggingface.co/docs/transformers/custom_models#writing-a-custom-model) instead of trying to change the model code.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.<|||||>Note that in `transformers` we don't want to necessarily support all the optimizations natively in the core library - also because the fundamental design is different (modeling code is copied rather than abstracted like in `diffusers`). 

Maybe also cc @michaelbenayoun here <|||||>Hi @cccntu,
I agree with @patrickvonplaten here. Also, note that we have a library for optimizations called [Optimum](https://github.com/huggingface/optimum). Altough we do not support custom optimization for each modeling, I think that you might be interested in contributing optimizations such as the ones on your PR there!<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.