Hello, everyone:
I faced a problem about reduction way of nll_loss, which give me extremely different result by same seed in GCN.
I am using GCN to do binary text classification. First, I use class weight in the nll_loss as belowed. It gives me good result. As I used same seed, everytime I rerun the GCN, I can get the same result. I can check the log each iteration of loss is same.
np.random.seed(args.seed)
torch.manual_seed(args.seed)
if args.cuda:
    torch.cuda.manual_seed(args.seed)

x_class = y.sum(dim=0).float()
z_weight = (1.0/x_class)*x_class.sum(0)/2.0
y = y.max(1)[1].type_as(labels)
return F.nll_loss(preds, y, z_weight)

But when I changed the nll_loss to reduction “none”, although the loss return is also same as above at the beginning, which value is 0.7351 at the first time, it will show different result after multi iteration at the end.
x_class = y.sum(dim=0).float()
z_weight = (1.0/x_class)*x_class.sum(0)/2.0
y = y.max(1)[1].type_as(labels)
loss_pre = F.nll_loss(preds, y, reduction='none')
final_weight = z_weight[y]
return (loss_pre * final_weight).sum() / final_weight.sum()

#original loss
Epoch: 0001
 | loss: 0.7351
Epoch: 0002
 | loss: 0.6355
Epoch: 0003
 | loss: 3.2794
...
Epoch: 0010
 | loss: 0.6107
...
Epoch: 0200
 | loss: 0.0207

#loss of reduction none
Epoch: 0001
 | loss: 0.7351
Epoch: 0002
 | loss: 0.6355
Epoch: 0003
 | loss: 3.2797
...
Epoch: 0010
 | loss: 0.6097
...
Epoch: 0200
 | loss: 0.0239

Could anyone give some help? Thanks indeed.
My environment is Python 3.6.9, pytorch 1.3.0 with CPU.