Solved by crcrpar in post #2 
              
              
                Hi, 
gradients are accumulated until model.zero_grad() is called. 
A toy is notebook is here: how-gradients-are-accumulated-in-real.ipynb · GitHub