Thank you for your replay!
But I still cannot understand what you mean by “it has 0 grad as it is symmetrical”. As far as I know, Normalize is a differential operation, and we can compute its gradient by Chain Rule.
And I just read the source code of F.normalize(), which is quite similar with my second code except the “detach()” operation.
def normalize(input, p=2, dim=1, eps=1e-12):
    return input / input.norm(p, dim, True).clamp(min=eps).expand_as(input)

Can you explain more clearly? Thank you!