Firstly, thank you so much @richard and @jpeg729 for the apt suggestions and the explanations for the same. I really appreciate you helping me resolve this issue. You were right, moving the optimizer outside the loop and the regularizer in fact does nothing!
I think the loss stays constant because of the normal k-means loss component that I calculate. The desired behaviour is when I use the pre-trained model that spits out some embeddings (X) which is converted to a variable like,
lstTxtOnly = 'text dataset'
R_init_numpy = 'some initial values'
R_init_means =  'some other initial values'
pre_R = Variable(torch.from_numpy(R_init_numpy + 1e-8).type(dtype), requires_grad=True)
U = Variable(torch.from_numpy(R_init_means).type(dtype), requires_grad=False)

F = torch.stack(Variable(torch.from_numpy(X).type(dtype), requires_grad=True))
where,
X = model.encode(lstTxtOnly, bsize=128, tokenize=False, verbose=False)
Then, the loss is computed over this variable F as.
distances = torch.sum(((F.unsqueeze(1) - U) ** 2), dim=2)
custom_loss = torch.sum(R * distances) / num_samples

But since the model parameters never update so the embedding value X also doesn’t change. F which was constructed from a numpy(X) won’t back-propagate.
How can I formulate this setup where the pre-trained model parameters are updated to spit out a new embedding that is then used to compute the custom loss and this is repeated for a fixed number of epochs?