Hi Learner!
I have a couple of comments and questions, in line, below:



 learner47:

There are N (say 10) classes to choose from for x.


Does this mean that you have 10 fixed values of x (say, 0.33, 0.52,
0.57, 0.68, etc.) that you know ahead of time?  Or is x continuous,
but you choose to break it up into 10 categories somehow?
If the latter, I would just predict x directly, and not (artificially) impose
10 categories on the problem.
If the former, I would still predict x directly – unless you are
purposely using 10 values / categories in order to set up a toy
classification problem with 10 categories.


self.softmax(self.fc5(self.fc4(self.relu(self.fc3(self.relu(self.fc2(self.relu(self.fc1(inputs)))))))))
...
mse_loss = nn.MSELoss()



Softmax and MSELoss don’t really go together.  Softmax would be
used for a classification (categorical) problem with a cross-entropy
loss (although you would actually leave out the Softmax and use
pytorch’s CrossEntropyLoss that has, in effect, Softmax built in).
MSELoss would be used for a regression-like problem (which is what
yours seems to be), but you would then predicts x directly, not pass
it through a Softmax, and not categorize it.


All the quantities in y = ax + b are vectors (each of length 4, in this example).


It looks to me as if your “vectors of length 4” make up batches of
batch size nBatch = 4, but that you are building your batch size
into your network (self.fc1 = nn.Linear(8, 76)).  Pytorch models
work implicitly with batches, so, with two input features (y and a), you
would want your network to start off with Linear (2, 76).  You would
then pass into your network a batch of shape
[nBatch, nFeatures = 2].  nBatch could be 4, but it could be other
values, say nBatch = 100, without changing the structure of the
network.
In short, I would have your network predict a single value of x (or more
precisely, a batch of single values of x, but pytorch does that for you
automatically), predict x directly (no Softmax or other non-linearity
following the final Linear layer).
So your network would return something like
return self.fc5 (self.relu (self.fc4 ( ... ) ) ), but with
self.fc5 = nn.Linear (14, 1).
(Note that self.fc5 (self.fc4 (...) ), without an intervening non-linearity
such as self.relu() acts as just a single Linear layer, not two.)
I would then use MSELoss as the loss criterion.
Best.
K. Frank