Solved by woaichipinngguo in post #3 
              
              
                It seems clear. Is this code snippet right? 
 
myNet=myNet.to(device)
sig1=torch.tensor(1.0,requires_grad=True, device=device)
sig2=torch.tensor(1.0,requires_grad=True, device=device)
criterion=nn.CrossEntropyLoss()

optimizer=optim.Adam(myNet.parameters(),lr=lr,weight_decay = opt.weight_decay)
sche…