optimizer.zero_grad() in run_openai_gpt.py?
In `run_openai_gpt.py`, should there be a call to `optimizer.zero_grad()` after updating parameters so that we zero out the gradients between minibatches?

https://github.com/huggingface/pytorch-pretrained-BERT/blob/2152bfeae82439600dc5b5deab057a3c4331c62d/examples/run_openai_gpt.py#L212
Oh that's a mistake indeed, thanks for pointing out.
Fixed on master.