Fix CLIP pooling for textual inversion so that eos tokens are taken
### Feature request

For textual inversion in diffusers, we are adding tokens that have a higher token id than the eos token. So when we get clip embeddings for textual inv tokens, we need to change the pooling so it gets the eos token and not the arg max token.



### Motivation

This is an issue that should be fixed as the clip embeddings won't work once we add more tokens to the tokenizer.

### Your contribution

I can make a pr for this. This is not an issue in the original implementation of the clip since they use pre-existing tokens in the embedding which has its own pro-cons.
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.