Solved by ptrblck in post #2 
              
              
                The tensor is normalized over dimension dim, such that: 
(x[:, 0, 0, 0])**2.sum() == 1
(x[:, 0, 0, 1])**2.sum() == 1
...

In your use case you could do the following: 
x_ = F.normalize(x.view(x.size(0), -1), dim=1, p=2).view(x.size())
(x_[0]**2).sum() == 1