enable mixed precision for Tensorflow training benchmarks
# 🚀 Feature request

Currently [Tensorflow Benchmarks](https://github.com/huggingface/transformers/blob/master/src/transformers/benchmark/benchmark_tf.py) implemented in transformers package only supports training in FP32 mode and FP16 support is [unimplemented](https://github.com/huggingface/transformers/blob/master/src/transformers/benchmark/benchmark_tf.py#L173). It could be helpful for the community to be able benchmark the training for the models in FP16 mode as using mixed precision greatly improves the performance of training.

## Motivation

Enabling mixed precision in training is [shown](https://medium.com/tensorflow/automatic-mixed-precision-in-tensorflow-for-faster-ai-training-on-nvidia-gpus-6033234b2540) to significantly improve the throughput of the training process. We wanted  to implement the missing support for fp16 for training in [Tensorflow Benchmarks](https://github.com/huggingface/transformers/blob/master/src/transformers/benchmark/benchmark_tf.py) to gauge the performance uplift we noticed for 1.5x improvement in performance for `bert-base-uncased` using batch size of `8` and sequence length of `128`.

## Your contribution

The code for this is implemented in [amp_tf_training_benchmarks](https://github.com/huggingface/transformers/compare/master...harishneit:amp_tf_training_benchmarks) branch in a fork. I can submit a pull request with tests if the community is interested in this.
Sorry for the slow reply - this is definitely something we'd be interested in seeing! Can I ask why you used `tf.compat.v1.mixed_precision.enable_mixed_precision_graph_rewrite` in your fork rather than `set_global_policy` or similar? It's not necessarily wrong, but I'm curious what the tradeoffs are there!<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.