Longformer: raise TypeError("pred must not be a Python bool", pred)
## Environment info

- `transformers` version: 4.2.2
- Platform: Ubuntu 18.04
- Python version: 3.7.6
- PyTorch version (GPU?): None
- Tensorflow version (GPU?): 2.3.1, 2.3.2 (with or without GPU)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help@patrick-s-h-lewis 

Models:
- longformer @patrickvonplaten

## Information
Errors occur when I use the TFLongformerMainLayer as a layer of my model. I will give a simple example below to reproduce this bug.

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce
Errors occurs in 'transformers/models/longformer/modeling_tf_longformer.py:1799 _pad_to_window_size  *
        inputs_embeds = tf.cond(padding_len > 0, pad_embeddings, lambda: inputs_embeds)'
It looks like `padding_len >0` is a python bool which caused this error. 
According to the [official guide of tf.cond](https://www.tensorflow.org/api_docs/python/tf/cond) example: `result = tf.cond(x < y, lambda: tf.add(x, z), lambda: tf.square(y))`, I think this is because both 'padding_len' and '0' are not tensors, so `padding_len >0` just returns a python bool.
```
TypeError: in user code:

    /home/xingya/.conda/envs/tf2/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py:806 train_function  *
        return step_function(self, iterator)
    test.py:44 call  *
        x = self.longformer(input_ids=None, inputs_embeds=inputs)[0]
    /home/xingya/.conda/envs/tf2/lib/python3.7/site-packages/transformers/models/longformer/modeling_tf_longformer.py:1680 call  *
        (
    /home/xingya/.conda/envs/tf2/lib/python3.7/site-packages/transformers/models/longformer/modeling_tf_longformer.py:1799 _pad_to_window_size  *
        inputs_embeds = tf.cond(padding_len > 0, pad_embeddings, lambda: inputs_embeds)
    /home/xingya/.conda/envs/tf2/lib/python3.7/site-packages/tensorflow/python/util/dispatch.py:201 wrapper  **
        return target(*args, **kwargs)
    /home/xingya/.conda/envs/tf2/lib/python3.7/site-packages/tensorflow/python/ops/control_flow_ops.py:1396 cond_for_tf_v2
        return cond(pred, true_fn=true_fn, false_fn=false_fn, strict=True, name=name)
    /home/xingya/.conda/envs/tf2/lib/python3.7/site-packages/tensorflow/python/util/dispatch.py:201 wrapper
        return target(*args, **kwargs)
    /home/xingya/.conda/envs/tf2/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py:507 new_func
        return func(*args, **kwargs)
    /home/xingya/.conda/envs/tf2/lib/python3.7/site-packages/tensorflow/python/ops/control_flow_ops.py:1180 cond
        return cond_v2.cond_v2(pred, true_fn, false_fn, name)
    /home/xingya/.conda/envs/tf2/lib/python3.7/site-packages/tensorflow/python/ops/cond_v2.py:62 cond_v2
        raise TypeError("pred must not be a Python bool", pred)

    TypeError: ('pred must not be a Python bool', True)
```
Here is a snipt to reproduce this bug:
```
from transformers.models.longformer.modeling_tf_longformer import TFLongformerMainLayer
from tensorflow.keras.layers import Input, Embedding, Dense
from tensorflow.keras.models import Model
from transformers import LongformerConfig
import tensorflow as tf
import numpy as np

tf.random.set_seed(200)


class LongFormerMain(tf.keras.layers.Layer):
    def __init__(self, name='longformer', **kwargs):
        super(LongFormerMain, self).__init__(name=name, **kwargs)
        config = LongformerConfig(attention_window=4, num_hidden_layers=1, vocab_size=10)
        self.longformer = TFLongformerMainLayer(config)

    def call(self, inputs):
        x = self.longformer(input_ids=None, inputs_embeds=inputs)[0]
        return x


inputs = Input(shape=(None,), dtype='int32')
output = Embedding(100, 768)(inputs)
longformer = LongFormerMain()
output = longformer(output)
output = Dense(9, activation='softmax')(output)
model = Model(inputs, output)
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy')

x = np.array([[5, 2, 3] * 3] * 100)
y = np.array([[1, 2, 3] * 3] * 100)
model.fit(x=x, y=y, epochs=10, batch_size=4, validation_split=0.1)
print(model.predict([[5, 2, 3] * 3]))
```

cc @jplu do you maybe have a good idea here?<|||||>Hey @xuxingya !! Thanks a lot for reporting the issue! Indeed Longformer has a bug in the `_pad_to_window_size` method. We will work on fixing this ASAP.

Even though there is indeed a bug, your piece of code is wrong and should be:
```python
from transformers.models.longformer.modeling_tf_longformer import TFLongformerMainLayer
from tensorflow.keras.layers import Input, Dense
from tensorflow.keras.models import Model
from transformers import LongformerConfig
import tensorflow as tf
import numpy as np

tf.random.set_seed(200)


class CustomLongFormer(tf.keras.layers.Layer):
    def __init__(self, name='longformer', **kwargs):
        super().__init__(name=name, **kwargs)
        config = LongformerConfig(attention_window=4, num_hidden_layers=1, vocab_size=10)
        self.longformer = TFLongformerMainLayer(config)

    def call(self, inputs):
        x = self.longformer(input_ids=None, inputs_embeds=inputs)[0]
        return x


longformer = CustomLongFormer()
inputs = Input(shape=(None, None), dtype='float32', name="inputs_embeds")
output = longformer(inputs)
output = Dense(9, activation='softmax')(output)
model = Model(inputs, output)
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy')

x = np.array([np.random.uniform(0,1, (3, 768))] * 100)
y = np.array([[1]*3] * 100)
model.fit(x=x, y=y, epochs=10, batch_size=4, validation_split=0.1)
```<|||||>Fixed in #9942 