Model templates tests are run twice
The CI currently runs the model template tests twice when opening a PR from a branch of the huggingface/transformers repo.

The `pull_request_target` should only trigger on external pull requests, or we should remove the `push` target so that the suite only runs once.

Additionally, the `pull_request_target` suite doesn't take into account the changes the PR is doing.

ETA until resolved ~ 2 weeks.