I am still not clear about " with torch.no_grad():"
with torch.no_grad():
            self.pred.weight.div_(torch.norm(self.pred.weight, dim=1, keepdim=True))

Is this means that the parameter of weight will not be update?

image.png1060×794 30.4 KB

Acutally, this is what I want. And I want to know how to get

param: weight which is not normalized
the weight which is used in fc calculation