Hi Nielsr,
I tried running the WNUT17 Trainer example in torch on both Kaggle and Colab TPUs, and neither seem to be working (although I made sure the TPUs were correctly configured and XLA was correctly installed).
Here’s the colab notebook: Google Colab 14
Here’s the Kaggle notebook: https://www.kaggle.com/xhlulu/huggingface-wnut17-tpu-tests?scriptVersionId=83062978 5
As you can see, each iteration takes significantly more time than it should be on GPU (total training time is ~1.5min on P100).