GLUE on TPU
This takes advantage of the pytorch 1.3 XLA implementation to fine-tune GLUE on a TPU.

MRPC fine-tuning in 3 epochs + evaluation takes a total of 6 minutes and 30 seconds.
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/1488?src=pr&el=h1) Report
> Merging [#1488](https://codecov.io/gh/huggingface/transformers/pull/1488?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/f382a8decda82062bb6911f05b646f404eacfdd4?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/1488/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/1488?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #1488   +/-   ##
=======================================
  Coverage   85.59%   85.59%           
=======================================
  Files          91       91           
  Lines       13526    13526           
=======================================
  Hits        11578    11578           
  Misses       1948     1948
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/1488?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/1488?src=pr&el=footer). Last update [f382a8d...639f4b7](https://codecov.io/gh/huggingface/transformers/pull/1488?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
<|||||>LGTM!<|||||>It is possible to save/load when on TPU, just move them to CPU and save them:

https://github.com/pytorch/xla/blob/master/API_GUIDE.md#saving-and-loading-xla-tensors