Yes, I do mean to just rerun optimizer=optimizer(model.params).  Make sure that if you have otimizer, and optimizer_1 its correct. and if any of these are being called in separate functions make sure that you are returning the new optmizer/model that you want to use from those functions.  I’m guessing the core of your problem is the same as mine, that when you call optimizer.step that optimizer is not actually pointing to your tensors that have the gradients.  but could be a number of ways that has happened.