Feature request: Adding einops as a Dependency
### Feature request

I propose that the einops library be added as a dependency in the HuggingFace Transformers library. einops (Einstein Notation Operations) is a Python library that provides a more expressive language for tensor operations. This would offer highly readable and maintainable code for complex tensor reshaping and rearranging operations.
einops GitHub page: https://github.com/arogozhnikov/einops

### Motivation

The addition of einops as a dependency would greatly facilitate the integration of new models that already use it into the Transformers library.  Adding einops would require less refactoring in adding these models, making the code easier to read, and will decrease the amount of time spent on understanding and maintaining the code.

Given the potential benefits that the inclusion of einops could bring to the Transformers library, and considering its impressive community support highlighted by over 7,000 stars on GitHub, I think that this proposal merits serious consideration. I suggest that we put this feature request up for a community vote. This would allow all contributors to weigh in on the decision. 

### Your contribution

.
There is actually a (recent) discussion

> There have been lots of discussions on this in the past, the TLDR is that we aren't supporting it cause it causes issues with things like ONNX

and the approach is more

>  just re-wrote that part in pure pytorch,

<|||||>For me there is no reason to add a dependency when `einops` would actually be detrimental to all the work done to optimize inference (quantization, ONNX etc.). We can revisit this if the support end up being on par with classic PyTorch operations, but we shouldn't make the code easier to read if it's not as efficient/supported by the ecosystem.<|||||>einops has torch.compile support, why should it be a incompatible with onnx?
https://github.com/arogozhnikov/einops/wiki/Using-torch.compile-with-einops<|||||>It might be the case some point in the past (I am not the one involved in this topic previously). Not sure the status at this moment. A further investigation into this to make sure the whole ecosystem will run smoothly with models using `einops` is necessary before we add it as a dependency. But the teams have their own priorities, and we would like to see how the community reacts to this feature request first. 