Thank you, Frank!
I now use

list_of_loss = []
...
for epoch in range(num_epochs):
    list_of_loss = [0 for _ in range(num_model)]

    for j, (images, labels) in enumerate(train_loader):
        for i in range(num_model):
            ...
            list_of_loss[i] = list_of_cri[i](output, labels)

            ...
            list_of_loss[i].backward()


To solve this.  Based on the numerical experiment, this seems to work. Is this correct?
My core question is: can we put all these terms into a list, use a list of optimizer/loss, update list_of_optimizer[i], backward the list_of_loss[i]? I wrote this demo code to numerically prove that.