I see, so in this case you can take the same approach but go into the opp direction of the gradient of the input features rather than the weights. I think that’s what you are doing now.

I will have:
y_pred = net(x)


however, this does not work for some reason. meaning, even after the ‘.backward()’ command the ‘.grad’ of ‘x’

Don’t know why it wouldn’t work, but you sure do have a lot of x’s there 
x = F.sigmoid(self.fc1(x))
x = F.sigmoid(self.fc2(x))
x = F.sigmoid(self.fc4(x))
x = self.fc3(x)
return x

To disentangle that a bit, I would make sure that you keep your input “x” distinct from the rest and see if that solves the problem