@InnovArul
I actually do intend on having params, params2 and params3 be optimized, and not just stored as intermediate placeholder variables. They culminate in an unsupervised loss function MyLoss. The gradients obtained via retain_grad() generate the partial derivatives MyLoss wrt params, MyLoss wrt params2 and  MyLoss wrt params3. Taking SGD as an example, the update for params2 would be:
params2 -= lr*(MyLoss wrt params2).
Where params2.data is set to what was during instantiation (before being affected by the intermediate pass). Ditto for the other params.
But again, I wish to abstract this away by using the optim class instance. I think I see what you mean by reassigning values creates different objects, as the optimizer loses track of params2 for example after the reassignment (please correct me if I’m wrong).
The reason why I reassign params* is because I wish to first instantiate them outside my training loop and pass them into an instance of a custom model I’ve built. Loosely speaking, what I’m trying to do is train a standard neural network, then freeze it (the trained fully connected weights), and train another model that takes these frozen weights and trains another set of parameters. These new parameters (the params*) act as activations similar to a fully connected net, but they are also trainable (they get corrected according to minimizing myLoss).
Please let me know if this is enough, and whether you can infer a solution to my problem. If not, I can provide additional details. I really appreciate your help!