Hello,
I’m now trying to use multiple models and optimizers in training.
During the training, it seems like the loss is not decreasing.
Can anyone know what the problem is?
The code is below
I have multiple models and optimizers, and in the for c in range(classnum):
loop, I am trying to compute the loss from each model and optimizing the model with optimizers corresponds to the model.
Note that each  optimizer is different alghorithm.
models = list[mode1, model2,...., model10]
optimizers = list[optimizer1, optimizer2,...., optimizer10]

for model in models:
            model.train()
            
for step, inputs in enumerate(tqdm(train_loader)):
     inputs = inputs.to(device)
            
     for c in range(classnum):
                
          optimizers[c].zero_grad()
                
                             :
                             :
                             :
           outputs, mean, logvar = models[c].forward(each_input)
           kld, recon = loss_function(outputs, each_input, mean, logvar, each_weights)
           loss = kld + recon
        
           loss.backward()
           optimizers[c].step()

           running_loss += loss.item()

Thank you.