Difference between FlaxViTModel and FlaxCLIPVisionTransformer
Hi, I noticed that HuggingFace has two different Flax-based implementations of the vision transformer, the [FlaxCLIPVisionTransformer](https://github.com/huggingface/transformers/blob/main/src/transformers/models/clip/modeling_flax_clip.py#L535) and the [FlaxViTModule](https://github.com/huggingface/transformers/blob/main/src/transformers/models/vit/modeling_flax_vit.py#L507). Is there a reason there is a separate ViT implementation for CLIP? Are there any notable differences between the FlaxCLIPVisionTransformer and the FlaxViTModel? Thank you!
Please use the [forums](https://discuss.huggingface.co/) for questions like this as we keep issues for bugs and feature requests only.<|||||>I think they're equivalent (one would need to check they are both applying pre-norm or post-norm etc.), but the reason we have 2 different implementations is because Transformers has a [one model, one file philosophy](https://huggingface.co/blog/transformers-design-philosophy).