[TensorFlow] Adding GroupViT
# Adding TensorFlow version of GroupViT

This PR adds the TensorFlow version of [GroupViT](https://github.com/NVlabs/GroupViT).

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


CC: @LysandreJik @Rocketknight1 
_The documentation is not available anymore as the PR was closed or merged._<|||||>Tagging @NielsRogge @Rocketknight1 @gante for the PR review!<|||||>@ariG23498 The parsing issue seen in the `check_repository_consistency` tests is arising because of `@keras_serializable` decorators being below `# Copied from` statements. If I check out your branch, I can run `make fix-copies` successfully if I move the decorator to be above the comment. <|||||>Hey @ariG23498 👋 

Regarding failing tests:
- The tests in `run_tests_torch_and_tf` seem to need a slightly higher tolerance. Let's leave those as their are for now (failing), as we might be able to fix them throughout the PR review process :)
- The other errors is because the attribute `base_model_prefix` does not match the name of the attribute that is used to hold the models. I.e. it should be changed to `base_model_prefix = "group_vit"` (notice the `_` :D )
[`base_model_prefix` wrong -> [this line](https://github.com/huggingface/transformers/blob/main/src/transformers/modeling_tf_utils.py#L1084) does not get matched with the correct class -> raises the exception in the function]
<|||||>Hey @gante thanks for the insights.

> The other errors is because the attribute base_model_prefix does not match the name of the attribute that is used to hold the models. I.e. it should be changed to base_model_prefix = "group_vit" (notice the _ :D )
[base_model_prefix wrong -> [this line](https://github.com/huggingface/transformers/blob/main/src/transformers/modeling_tf_utils.py#L1084) does not get matched with the correct class -> raises the exception in the function]

The [PyTorch](https://github.com/huggingface/transformers/blob/8cb5ecd912e09301be126c6ce6e9a22ca7153da4/src/transformers/models/groupvit/modeling_groupvit.py#L769) implementation has `groupvit` as the `base_model_prefix`. Would this be a problem if I changed the name for the TensorFlow port?
<|||||>@ariG23498 Good point -- `groupvit` (no underscore) should be used everywhere then!<|||||>Fixes #18543 

CC: @NielsRogge <|||||>References
- hard_softmax: https://gist.github.com/ariG23498/08cdae21637b8b61bdd6d21d11719fb3
- resize_attention_map: https://gist.github.com/ariG23498/3777f8d9be25de8ae782256f5aacb2c5<|||||>@amyeroberts I have added the `shape_list` back to places which are copied from other parts of the repository. This was done to pass the `make fixup`.<|||||>Hey @amyeroberts and @gante I had to swap back in the `shape_list` and also the `axis=range(len(shape_list(logits)))[dim]` to pass the tests.<|||||>Perhaps relevant to the TF-PT mismatch: https://github.com/huggingface/transformers/pull/18555#issuecomment-1230066514<|||||>> Perhaps relevant to the TF-PT mismatch: [#18555 (comment)](https://github.com/huggingface/transformers/pull/18555#issuecomment-1230066514)

Hey @gante I do not think this might be the source of the problem.

`nn.functional.interpolate` has been used twice in the torch code:
- [`resize_attention_map`](https://github.com/ariG23498/transformers/blob/feat/groupvit-tf/src/transformers/models/groupvit/modeling_groupvit.py#L135)
- [`interpolate_pos_encoding`](https://github.com/ariG23498/transformers/blob/feat/groupvit-tf/src/transformers/models/groupvit/modeling_groupvit.py#L416)

I have a [colab notebook](https://gist.github.com/ariG23498/3777f8d9be25de8ae782256f5aacb2c5) that takes care of the tf and pt equivalence for the `resize_attention_map`. As for the `interpolate_pos_encoding`, it has been taken from the ViT implementation (which is assumed to have no equivalence problem?).<|||||>@ariG23498 With the selected seeds, it passes now on CircleCI. This depends also on hardware, so it may or may not work when running on different machines, sadly.<|||||>It passes on GCP CPU-only VM too. For GPU, we might need different seeds, but we can do that later.<|||||>`tests/models/groupvit/test_modeling_tf_groupvit.py::TFGroupViTModelTest::test_keras_fit` failed however. I will leave @ariG23498 to check this one 🙏 <|||||>Tagging @sgugger for a review (as core maintainer) 🎉 

After this PR has 3 approvals (including Sylvain's), these are the instructions to finalize the PR:
1. Add the TF weights for the model(s). Assuming you are logged in to the Hub in your terminal (if not, run `huggingface-cli login`) and are on this branch, run `transformers-cli pt-to-tf --model-name foo/bar`, where `foo/bar` is the target model repo. Assuming it passes the CLI's checks, it will open a PR on the model hub. Please tag `@_nielsr, @_sgugger, @_joaogante` (⚠️ removing the `_`) in the opened Hub PRs :)
2. After the TF model weights are merged, make sure there are no `from_pt=True` in the PR. Then, re-run the test suite for the model locally and confirm that all tests pass (command: `NVIDIA_TF32_OVERRIDE=0 RUN_SLOW=1 py.test -vv tests/models/groupvit/modeling_tf_groupvit.py`)<|||||>The [`run_tests_hub`](https://app.circleci.com/pipelines/github/huggingface/transformers/47547/workflows/26d9127c-5aac-43e9-99ff-8a42a25aa0e1/jobs/563161) fails for the time being. @ydshieh thinks it is irrelevant to this PR. All the relevant tests pass.

Pinging @Rocketknight1 for the serving outputs bit (as mentioned by @sgugger [here](https://github.com/huggingface/transformers/pull/18020#pullrequestreview-1112320572)).<|||||>@Rocketknight1 The changes you have suggested are taken care of! @ydshieh the tests pass as well!

@gante @amyeroberts should we talk about the deprecated code used [here](https://github.com/huggingface/transformers/pull/18020#discussion_r974312323), or should we have a different PR to handle this?

@gante over to you for saving the TF model to hub 🫂<|||||>FYI: The failing tests are not related to the `GroupViT` model.<|||||>The failure is unrelated. As long as all models have been moved this PR can be merged.<|||||>@gante over to you now! 🤗 <|||||>@ariG23498 adding TF weights as PRs to the hub is now available to everyone -- you can have a go at it, following [the instructions above](https://github.com/huggingface/transformers/pull/18020#issuecomment-1250738894)! 

I can also do it. Let me know how you want to proceed :)<|||||>Thanks @gante I will give it a try myself!<|||||>@gante while running `transformers-cli pt-to-tf --model-name nvidia/groupvit-gcc-yfcc` I get the following error.

```
List of maximum hidden layer differences above the threshold (5e-05):
text_model_outputhidden_states[1]: 9.155e-05
text_model_outputhidden_states[2]: 9.155e-05
text_model_outputhidden_states[3]: 9.155e-05
text_model_outputhidden_states[4]: 9.155e-05
text_model_outputhidden_states[5]: 9.155e-05
text_model_outputhidden_states[6]: 9.155e-05
text_model_outputhidden_states[7]: 9.155e-05
text_model_outputhidden_states[8]: 9.155e-05
text_model_outputhidden_states[9]: 9.155e-05
text_model_outputhidden_states[10]: 9.155e-05
text_model_outputhidden_states[11]: 9.155e-05
text_model_outputhidden_states[12]: 9.155e-05
vision_model_outputhidden_states[1]: 2.441e-04
vision_model_outputhidden_states[2]: 7.629e-05
vision_model_outputhidden_states[3]: 7.629e-05
```

I think this is because of the seed issue that @ydshieh had pointed out.

How would you want me to proceed in this case?<|||||>@gante thanks for the help.

The PR for the TF weights on HF Hub are [here](https://huggingface.co/nvidia/groupvit-gcc-yfcc/discussions/1).<|||||>It seems ready to be merged, except for an inconsistency that arises from a `# Copied from` statement -- lmk if you need a hand!<|||||>@gante I have taken care of the `copied from` inconsistency.

I think we are good to go.<|||||>Merged! Thank you for making the TF users happier, @ariG23498 🧡 