if x.norm or w.norm is 0, then you essentially have 1e-5 in the denominator.
If you pass that gradient through a couple of BatchNorm layers’ backwards, then it might start generating NaNs, and not at the CosNorm itself.