Hi Saida!



 Saida2020:

during the training, the loss values start to have numbers then inf then NAN.


Because you are performing a regression with MSELoss, your
model should not be actively unstable.  However, depending on
data, your model could be underdetermined – especially if you
have fewer samples than parameters – and therefore also might
not be actively stable.
I would suggest you add weight_decay to your SGD optimizer.
This is equivalent to adding an “L2” penalty to your loss function,
and should keep any underdetermined parameters from running
off to infinity.
Some side comments:


I have classification problem. My input is sequence of length 341 and output one of three classes {0,1,2}


If you really have a classification problem, you should consider not
trying to approach it as a regression.  Do your class labels really
represent something numerical, such as $0, $1, $2, where “1” is
numerically between “0” and “2” (and indeed, halfway between)?
Or are your labels just arbitrary class labels, where, for example,
“0” means “dog,” “1” means “cat,” and “2” means “horse?”
If your labels are not naturally numerical, you should treat your
problem as a classification problem.  You would want to add more
layers to your model, separated by non-linear “activations,” and use,
most likely, CrossEntropyLoss as your loss function.


            X = Variable(X, requires_grad=True)
            y = Variable(y, requires_grad=True)



There is no reason to set requires_grad = True for X and y.  Any
gradients that you calculate for X and y just get ignored.  (X and y
haven’t been added to your optimizer – nor should they be.)
Also, Variable is deprecated – it doesn’t really do anything anymore.
(To set requires_grad = True for X – but you don’t want to – just
use X.requires_grad = True.)
Best.
K. Frank