Longformer Set Global Attention Layers Non-Trainable
# 🚀 Feature request

Tensorflow Longformer model set global attention layers non-trainable.

## Motivation

During training global attention is not used giving zero gradients for all global attention weights. Setting the global attention layers to non-trainable will decrease training time and reduce memory consumption.

Hi @MarkWijkhuizen! Thank you for your input. 

If you have the time, I'd like to encourage you to open a PR with the change. Otherwise, no worries, let us know (and, if possible, give us more pointers -- e.g., a reference to the paragraph in the paper where they mention this part of the training regime) 🤗 <|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.