dzimm:

torch.randn(64, 512, 1, 1).normal_(mean=0.0007720401627011597, std=0.01006225310266018))


I am running this on pytorch 0.4 and I haven’t seen any nans yet. @isalirezag what version of pytorch are you on?