Hi, I’m trying to modify the mean/std of one feature with the mean/std calculated from another feature. It looks like this (certain simplification is made since original code is much more complicated)

def exchange(vs, vt):
    # vs and vt are of the same size NxCxHxW
    vs_mean = torch.mean(vs, dim=(2, 3))
    vs_std = torch.std(vs, dim=(2, 3)) + self.eps
    raw_es_domain = torch.cat((vs_mean, vs_std), dim=1)
    es_domain = self.relu(self.fc1(raw_es_domain))
    es_domain = self.relu(self.fc2(es_domain))

    weight = self.fc_w(es_domain) # size of NxCx1x1
    bias = self.fc_b(es_domain) # size of NxCx1x1

    vt = nn.InstanceNorm2d(vt.size(1), affine=False)(vt)
    vt = weight * vt + bias

    return vt

However, there is this nan bug that happens sometimes, while sometimes this code snippet works just fine. I’ve used torch.autograd.detect_anomaly() to debug, and it told me the  torch.std operation was the culprit. The simplified log information is listed as follows:
/opt/conda/conda-bld/pytorch_1573049306803/work/torch/csrc/autograd/python_anomaly_mode.cpp:57: UserWarning: Traceback of forward call that caused the error:
vs_std = torch.std(vs, dim=(2, 3)) + self.eps
loss.backward()
File "/home/user/anaconda3/envs/pytorch-1.3.1/lib/python3.7/site-packages/torch/tensor.py", line 166, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/home/user/anaconda3/envs/pytorch-1.3.1/lib/python3.7/site-packages/torch/autograd/__init__.py", line 99, in backward
    allow_unreachable=True)  # allow_unreachable flag

But I’ve already added an eps to the vs_std. Is there any way to fix this bug? Or is there any other way to find out what’s the cause of nan? Thanks!