Dear all,
I have a very simple question about the gradient flowing backward through the InstanceNorm2d layer.
Here are my test codes:
x = torch.arange(0., 8).reshape((2, 1, 2, 2))
x.requires_grad = True
instaceN = nn.InstanceNorm2d(1, affine=False, eps=0.0, track_running_stats=False)
instaceN.weight = nn.Parameter(torch.Tensor([1.0]))
instaceN.bias = nn.Parameter(torch.Tensor([0.0]))
y = instaceN(x)
y.register_hook(print)
z = y.sum()
z.backward()
print(x.grad)

But the output of x.grad is all 0s. I am wondering how this could happen?
If I change InstanceNorm2d to BatchNorm2d, x.grad is sill all 0s.
But if I change InstanceNorm2d to Sigmoid, x.grad has none-zero grads.
Could someone explain this weird phenomenon?
Thanks!