Remove hard-coded uses of float32 to fix mixed precision use in Distilbert
In this commit [Remove hard-coded uses of float32 to fix mixed precision use](https://github.com/huggingface/transformers/commit/4fca874ea995f3d23ad7062b07b5ed7c4f87c0cd#diff-e3ab4f29f29fe1d243a6b55fafaab097), the mixed precision issue is fixed for [modeling_tf_bert.py](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_tf_bert.py).

However, for [modeling_tf_distilbert.py](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_tf_distilbert.py#L171), the line 171 is not fixed yet, and we get 

>    173         embeddings = inputs_embeds + position_embeddings  # (bs, max_seq_length, dim)
>    --> 174         embeddings = self.LayerNorm(embeddings)  # (bs, max_seq_length, dim)
>    InvalidArgumentError: cannot compute AddV2 as input #1(zero-based) was expected to be a bfloat16 tensor but is a float tensor

while using `mixed_bfloat16 mixed precision` with `TPU`.

A very quick fix is the same as the fix for `modeling_tf_bert.py`:

    position_embeddings = tf.cast(self.position_embeddings(position_ids), inputs_embeds.dtype)

@schmidek 
Indeed! Do you want to open a PR to fix this?<|||||>@LysandreJik 

I can do that. However @patrickvonplaten  has already self-assigned for this. How do you think, @patrickvonplaten?<|||||>Hey @chiapas, it would be great if you can open a PR for it :-) <|||||>Hi @patrickvonplaten , OK, that would be my first contribution to transformers :)