gradient of torch.norm at 0 (in version 0.2 and before) is NaN.
It might be that the backward of any of the norm ops is generating nan.
We fixed this in master, where the gradient for norm uses subgradients and hence grad(norm(0)) = 0, it will be part of next release (or you can try it by building PyTorch from source: https://github.com/pytorch/pytorch#from-source 54 )