Hi all,
I have a weird problem (so I guess). I want to train a model many times on different elements of the output. This works only if I output the respective element directly and it doesn’t work if I output a vector with all elements and calculate the loss only on one.
Lets say I have an output tensor r with size 1x4 and I’m interested in the loss on the last element r[3].
If I return from my forward path:
return r[3]
And then train:
y_hat = model(X)
loss = criterion(y_hat, y)
The predictions on the test set are perfectly fine. But if i return all elements:
return r
And train:
y_hat = model(X)
loss = criterion(y_hat[3], y)
In the ladder case the prediction is way off. In debugging mode, I see that the value which is passed to the loss function is in both cases exactly the same. Do I miss something here?
Thank you very much for your help!
Greetings