thanks!
by ‘disentangle’, do you mean something like this:
def forward(self, x):
    y = F.sigmoid(self.fc1(x))
    y = F.sigmoid(self.fc2(y))
    y = F.sigmoid(self.fc4(y))
    y = self.fc3(y)
    return y
have tried it. didn’t help.
here is something I’ve discovered which may shed some light on this problem:
the output also remains with ‘grad=None’, even though it also has ‘requires grad = true’.
…
edit:
I was given some more good advice and decided to do it like this:
grad = torch.autograd.grad(loss, input)
grad = torch.tensor(grad[0])
input -= learning_rate * grad
which works, only now I get another problem:
my input need to go through a softmax function (needs to be all positive and with a sum of 1)
and for some reason it always becomes a one-hot-vector. (one element equals 1 and all the others 0)
which makes the softmax function output ‘Nan’.
every time it’s a different number which becomes 1 with all the others being zero.