Hi Wei!



 weiguowilliam:

list_of_loss = []
...
for epoch in range(num_epochs):
    for j, (images, labels) in enumerate(train_loader):
        for i in range(num_model):
            ...
            list_of_loss.append(loss)
            ...
            list_of_loss[i].backward()

However, if I comment list_of_loss.append(loss) and list_of_loss[i].backward(), but use loss.backward() straightforwardly in each loop, there’s no error.


Your list_of_loss keeps growing with every iteration of the loops
over num_epochs and train_loader.
Therefore you keep calling list_of_loss[0].backward() on the
same loss over and over again that was created for the first sample
in your first epoch.
Put another way, the newly appended loss could be accessed and
backpropagated as:
list_of_loss[-1].backward()

which is not the same as:
list_of_loss[i].backward()

Best.
K. Frank