Hi, I am trying to add noise to layers’ output. I take the code from stylegan2-pytorch 1.
I thought the self.weight is a learnable parameter thus it can update during training since it is nn.Parameter(), but when I track the value of grad and weight of this layer the weight remains 0 and grad is None from the beginning. Can any one give me suggestions about this? thanks.
class NoiseInjection(nn.Module):
    def __init__(self):
        super().__init__()

        self.weight = nn.Parameter(torch.zeros(1), requires_grad=True)

    def forward(self, x, noise=None):
        if noise is None:
            batch, _, height, width = x.shape
            noise = x.new_empty(batch, 1, height, width).normal_()
        return x + self.weight * noise