Hi, I have some issues about autograd. I try to implement a network whose nodes in the hidden layer takes diverse noisy inputs. However, it hindered when computing the gradients w.r.t all the parameters.
num_hid = 1
nr_classes = 10
w1 = torch.rand(1024,num_hid,requires_grad = True)-0.5
w1_bias = torch.rand(num_hid,requires_grad = True)-0.5
w2 = torch.rand(num_hid,nr_classes,requires_grad = True)-0.5
w2_bias = torch.rand(nr_classes,requires_grad = True)-0.5
noise_gt_data1 = GenerateNoisyData(n)
linear_output_first_layer = torch.matmul(noise_gt_data1.reshape(-1,1024),w1)+w1_bias
sigmoid = nn.Sigmoid()
activation_output_first_layer = sigmoid(linear_output_first_layer)
out = Variable(torch.matmul(activation_output_first_layer,w2),requires_grad=True)+w2_bias
net = Net(num_hid)
net.body[0].weight.data = w1
net.body[0].bias.data = w1_bias
net.fc[0].weight.data = w2
net.fc[0].bias.data = w2_bias
y = criterion(out, gt_onehot_label)
gradients=torch.autograd.grad(y,net.parameters(),create_graph=True,retain_graph=True,allow_unused=True)
print(gradients)
It shows gradients are all None. I tried both backward() and autograd.grad(). Neither works.