That post helps me a lot. But i am still a little confused.
First, is it means that if the batch_size of train_loader = 1, i can write the code like this? because of accumulated gradient as you said?
criterion = nn.CrossEntropyLoss(）
for epoch in range(num_epochs):
    optimizer.zero_grad()
    for i, data in enumerate(train_loader, 0):
        inputs, labels = data
        output_a, outputs_b = net(inputs)
        loss_a = criterion(outputs_a, labels)
        loss_b = criterion(outputs_b, labels)
        (loss_a + loss_b).backward()
    optimizer.step()

Second, if the batch_size of the train_loader more than 1, e.g.16. i just change the criterion like this, and will get the same result?
criterion = nn.CrossEntropyLoss(size_average=False, reduction='sum')

Loss_total = $\sum_{i=1}^{N}(loss_a_i + loss_b_i)$. is waht i want to minimize.
N is the number of examples in my dataset.
 Thanks~