Hi Igor!



 igkrk:

        loss = ce(y_pred, y_train)
        loss_mse = mse(y_pred, y_train)



Your problem is that CrossEntropyLoss and MSELoss work rather
differently (both conceptually and mechanically).
CrossEntropyLoss expects its input (your y_pred) to be a set of
logits (see below) for the classes your model outputs, typically of
shape [nBatch, nClass] (in your case [100, 10]), and its target
(your y_train) to be integer class labels, typically of shape [nBatch]
(no nClass dimension) with values that range from 0 to nClass - 1.
In contrast, MSELoss expects its input and target to have the same
shape as one another and uses mean-squared-error to measure, on
average,  how close the individual elements of input and target are
to one another.  You are passing an input of shape [100, 10] and a
target of shape [100] to MSELoss, hence the error.
But furthermore, conceptually, a network that is trained to perform
classification outputs logits (or other probability-like values) for each
of the classes.  In the typical use case, these values are not naturally
comparable to other numbers, which is to say, using them with
MSELoss doesn’t really make sense.
You need to think about the meaning of your y_train values (and
similarly y_pred) and whether it even makes sense to use them
with MSELoss.


        loss.backward()
        loss_mse.backward()



As an aside, it’s a little more efficient to go backward() through your
network just once (per optimizer step), so you could rewrite this as
        loss_total = loss + loss_mse
        loss_total.backward()
        # or just  (loss + loss_mse).backward()



        return f.log_softmax(X, dim=1)



Note that CrossEntropyLoss has log_softmax() built into it, so you
don’t want this.  CrossEntropyLoss takes raw-score logits that run
from -inf to inf and are typically just the output of your final Linear
layer.
If you want that final log_softmax() you should use NLLLoss instead
pf CrossEntropyLoss.  (CrossEntropyLoss is just log_softmax()
and NLLLoss put together for you.)
Best.
K. Frank