Fix nightly CircleCI
# What does this PR do?

Fix a few nightly CircleCI issues

The effect could be found [here](https://app.circleci.com/pipelines/github/huggingface/transformers/50144/workflows/aaa65460-bbb5-4048-af3e-9450af06e231).
_The documentation is not available anymore as the PR was closed or merged._<|||||>@sgugger I accidentally push `GitPython` to `main`

https://github.com/huggingface/transformers/commit/6f8064da6b0c8f003731f292acc64f281a2aea65

😨 <|||||>Ah so this is done, perfect!