As a first debugging step you could make sure that the same parameters are properly registered and returned in model.parameters() in both approaches. Once this is done, make sure you are not freezing some parameters differently (if that’s even used). If this is also verified, I’m unsure why the optimizer would not perform the same updates, so you would need to explain this concern a bit more.