Hi guys I’m implementing a custom pooling function where there is a trainable tensor. So far I’m using Variable to wrap the tensor and requires_grad=True. However, it seems the that the variable is not changing as the model trains, it keeps stuck at the init value of 3. What am I doing wrong?
Here is the code:
class GeneralizedMeanPooling(nn.Module):
    def __init__(self):
        super(GeneralizedMeanPooling, self).__init__()
        self.p = torch.autograd.Variable(torch.cuda.FloatTensor([3]),
                                         requires_grad=True)

    def forward(self, x):
        B, C, H, W = x.shape
        x = x.view((B, C, -1))
        x = torch.sum(x ** self.p.expand_as(x), dim=2) / (H * W)
        x = x ** (1 / self.p)
        return x

Thanks!