I appreciate your reply
In this case
torch.autograd.backward([output],[grad])

Is grad of each data in minibatch accumulated like bellow?

  
    
    
    How to implement accumulated gradient in pytorch (i.e. iter_size in caffe prototxt) 
  
  
    how to can i accumulate gradient during gradient descent in pytorch (i.e. iter_size in caffe prototxt). 
Currently, my code is: 
     for iter, (images, labels, indices) in enumerate(train_loader, 0):
 
            optimizer.zero_grad() 
            outputs = net(Variable(images.cuda()))
            loss    = criterion(outputs, Variable(labels.cuda()))
            loss.backward()
            optimizer.step()

Do i do this? 
     for iter in range(N):
 
            optimizer.zero_grad() 
        …
  


Thanks