Thank you very much, just tried it and it improves the performance by almost 8%!
I am also able to use a much more reasonable learning rate (0.001, i will try with learning rate decay as well) !
Why is it so important to zero gradients ? Is that because it would otherwise compute gradients and add them addition to those from the previous iteration (sorry… not sure of the meaning of accumulating here) ?
Can you imagine a situation were you would not do such a thing ? From what I have seen, it could be implicit when using optimizer.step()
I am already using 0.4.0 release 
Thank you again !