Solved by ptrblck in post #2 
              
              
                As a first debugging step you could make sure that the same parameters are properly registered and returned in model.parameters() in both approaches. Once this is done, make sure you are not freezing some parameters differently (if that’s even used). If this is also verified, I’m unsure why the opti…