Then self.centers does not need to be a nn.Parameter() and can be defined as self.centers = torch.randn(num_class,num_dim).cuda().
You can then update it by just doing one of the two below
# works only if self.centers.requires_grad == False
self.centers.copy_(centers_update.detach())

# Will work in all cases
with torch.no_grad():
    self.centers.copy_(centers_update)