Ignore me!
I did this PR by mistake, and found out late. Sorry Hugging Face!
Sorry for the mess-up, was working on a fork of mine (facepalm) and accidentally PR to the repo.<|||||>No worries – but potentially open to a PR adding Azure logging if it makes sense on your end!<|||||>Hi @julien-c , thanks for pointing that out!

I am running `transformers` on AzureML (finetuning on classification tasks mainly), as I find it convenient because of the e.g. automatic compute provisioning and downsizing, experiment tracking, model versioning capabilities Azure offers.

I don't know what's the best way to contribute with respect to this (i.e. sharing code running finetuning of BERT using AzureML), but here's what I know:

- Implementing tensorboard-like capabilities in AzureML experiments can be done with trivial modifications to the current `trainer.py`, along the lines of https://github.com/huggingface/transformers/commit/939f6798acb178e3d8388e4597ef7e3dfcef0bf2#diff-cd3b4dd7d09c0b32ff40ccc1840a4da4

- Those trivial changes allow to see in the AzureML frontend trends like

![image](https://user-images.githubusercontent.com/4547987/93239437-0ed77880-f783-11ea-803e-f674a378f611.png)

which can be handy when running experiments.

- To launch one experiment, one would use Azure-specific code like that in https://github.com/microsoft/AzureML-BERT/blob/master/finetune/PyTorch/notebooks/Pretrained-BERT-GLUE.ipynb

Microsoft had started a pretty cool repo at https://github.com/microsoft/AzureML-BERT/ with solutions for this using https://github.com/huggingface/transformers/tree/v0.6.2 . What I think is happening though is that `transformers` has evolved quite a lot since then and they would need to refresh those examples.
 
I honestly don't know if it'd make sense to PR https://github.com/microsoft/AzureML-BERT (and not clutter the `transformers` with platform-specific code). WDYT? Cheers!
<|||||>The idea above is resurrected here https://github.com/huggingface/transformers/pull/8062 