Mask t5 relative position bias then head pruned
# What does this PR do?

Fixes #17886

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@patrickvonplaten

_The documentation is not available anymore as the PR was closed or merged._<|||||>Thanks for the PR @hadaev8! 

Could you add a test for the newly added functionality ? :-) <|||||>@patrickvonplaten 
Never did it, how it should looks like?<|||||>Hey @hadaev8,

The test should be added to this file here: https://github.com/huggingface/transformers/blob/main/tests/models/t5/test_modeling_t5.py

In this test it would be great if you could do the following for example:
Create a dummy T5 model and run a forward pass with `output_attentions=True`. The prune a head and run a forward pass again with `output_attentions=True`. Then you can compare that the attentions returned by the second forward pass will be 0 or just have fewer tensors because the head was pruned<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.<|||||>@patrickvonplaten 
I spotted another thing.
In encoder-decoder only one broken function to prune. Should I split in two?<|||||>@hadaev8 let's maybe do this in another PR :-) <|||||>Okay, i only will make test<|||||>Hey @hadaev8, 

Sorry last thing - could you maybe remove the accidently added `datasets` folder? See: https://github.com/huggingface/transformers/pull/17968/files#diff-714284abfa95a1447d7c34554c2d65b16fcfb1af22a44fc15489d13b76e951e5<|||||>@patrickvonplaten 
Sorry, still had no time to write the test.
Removed datasets folder.<|||||>It seems there is an issue with your CircleCI permissions, the tests won't run.
Could you try refreshing your permissions as shown [here](https://support.circleci.com/hc/en-us/articles/360048210711-How-to-Refresh-User-Permissions-)?<|||||>Seems I can't register account at CircleCI because sanctioned country.<|||||>Arf that's super annoying, sorry about that @hadaev8. I'll look into triggering it for you.<|||||>I pushed the same commits under a different branch: https://github.com/huggingface/transformers/tree/fix_t5_pruning-lysandre
It used my token permissions so it could run. Sorry you're experiencing this, I'll handle the triggers if some tests need to be fixed.<|||||>All tests pass, thank you @hadaev8! Merging the PR.<|||||>@LysandreJik 
Cool, thank you.