Gradient norm clipping should be done right before calling the optimiser
Right now it's done after each step in the gradient accumulation. What do you think?
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/1580?src=pr&el=h1) Report
> Merging [#1580](https://codecov.io/gh/huggingface/transformers/pull/1580?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/82f6abd98aaa691ca0adfe21e85a17dc6f386497?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/1580/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/1580?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #1580   +/-   ##
=======================================
  Coverage   86.16%   86.16%           
=======================================
  Files          91       91           
  Lines       13593    13593           
=======================================
  Hits        11713    11713           
  Misses       1880     1880
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/1580?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/1580?src=pr&el=footer). Last update [82f6abd...abd7110](https://codecov.io/gh/huggingface/transformers/pull/1580?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
<|||||>Oh yes, great, thanks, Pasquale. Would you mind fixing the `run_glue` and `run_ner` examples as well?<|||||>@thomwolf done! what's the best way to check this code before merging?<|||||>Thanks a lot!
It should be fine, we have continuous integration tests on `run_glue` and `run_squad` so if it passed at least the code run.