I trained the mode in mixed precision. in certain function, a input tensor in float16 is transformed into float32 after torch.norm
截屏2021-02-04 下午4.36.25974×688 101 KB
dis_vec (float16)  dis (float32)
and if I use dis=torch.sqrt((dis_vec**2).sum(-1)), dis is also float32