how to can i accumulate gradient during gradient descent in pytorch (i.e. iter_size in caffe prototxt).
Currently, my code is:
     for iter, (images, labels, indices) in enumerate(train_loader, 0):
 
            optimizer.zero_grad() 
            outputs = net(Variable(images.cuda()))
            loss    = criterion(outputs, Variable(labels.cuda()))
            loss.backward()
            optimizer.step()

Do i do this?
     for iter in range(N):
 
            optimizer.zero_grad() 
            loss = 0

            for i in range(M):
                  (images, labels, indices)=train_loader.next():
                   outputs = net(Variable(images.cuda()))
                   loss    +=  criterion(outputs, Variable(labels.cuda()))
                   loss.backward()

            optimizer.step()
            loss = loss/M