Is it possible to fine-tune models on TPUs using TensorFlow?
I have looked at the release notes and found out that:

"Training on TPU using free TPUs provided in the TensorFlow Research Cloud (TFRC) program is possible but requires to implement a custom training loop (not possible with keras.fit at the moment).
We will add an example of such a custom training loop soon." (Note from September 26).

Is this observation still true? Can we train the transformer models on TPUs in TF?
following, would love to know if this is possible<|||||>We have some code in the `tpu-experiment` branch, for instance here: https://github.com/huggingface/transformers/tree/tpu-experiments/examples/TPU/tensorflow

And planning to make it clean in the mid-term (not sure that will be before the end of the year though).

cc @LysandreJik <|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>Any updates on this one?<|||||>Same here<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>Same here <|||||>bump! Would love this! If you guys have your hands full, let me know if I can help in anyway :)<|||||>Hi! We recently have updated all of our scripts with `Trainer` classes, for both TensorFlow and PyTorch. Both trainers now have TPU support!

The [examples README](https://github.com/huggingface/transformers/tree/master/examples#running-on-tpus) has been updated accordingly.<|||||>That's great! Will try it out and report! <|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
