I tried 2 kinds of computing multi-losses methods in the deeply-supervised nets.
The first one needs to add retain_graph=True in loss.backward() while the second doesn’t and I don’t understand why.
...
output = model(data)
loss = 0
for i, out in enumerate(output):
    ...
    loss+=(F.nll_loss(out, target))
loss.backward()

...
output = model(data)
loss = 0
loss1 = []
for i, out in enumerate(output):
    ...
    loss1.append(F.nll_loss(out, target))

loss = reduce(lambda x, y: x+y, loss1)
loss.backward()