Solved by ptrblck in post #3 
              
              
                Both approaches should result in accumulated gradients. 
Since the gradients are accumulated, you might need to lower the learning rate or scale the losses.