Solved by yunusemre in post #3 
              
              
                I don’t know if there is a function which update different layers with different losses, but i think you can try like this : 
# First, update first layer
for i, param in enumerate(model.parameters()):
    if i != 0:
        param.requires_grad = False
    else:
        param.requires_grad = True
los…