Hi Learner!



 learner47:

My xs are known before hand. It can be, for example, binary representation of 0 to 10 (each representation having length 4).


I would say that I still don’t really understand your use case.
Your xs:  Would you say conceptually that they represent classes
(categories) that, although they can be labelled or encoded with
numbers, are not really numerical, but are something like “cat”,“dog”,
“goat”, where there is not really a sense of “dog” being in between
“cat” and “goat”.  Or are your xs fundamentally numerical, in the sense
that 2 is closer to 1 than it is to 4?
In the first case, you would want to understand your problem as a
multi-class (presumably ten-class) classification problem, have your
model output ten logits from its final Linear layer, and use (something
like) CrossEntropyLoss as your loss criterion.
In the second case, you would want to understand your problem as
a regression problem, have your final Linear layer output a single
predicted numerical value, and use (something like) MSELoss as
your loss criterion.
The underlying question is what, conceptually, do your xs represent?
You talk about x being a “representation of one of the 10 classes,” so
I tend to think that x, although perhaps a number or set of numbers,
encodes a non-numerical category.


The sizes of y, a, x and b are all (batch_size, 4).


Understood.  4 is not the batch size.
So it sounds to me like you have one x that is encoded somehow with
four numbers, and four randomly chosen as and bs that you use
with the four pieces of x to produce four ys.  The four ys and four
as are together input to your network, so, indeed, one input sample
consists of eight numbers.  (The four bs are not input to your model,
so they are in some sense “hidden” or noise.)
That’s fine.  The question – to repeat – is what is the conceptual
meaning of x (regardless of how it is encoded), so should you be
performing a (ten-class) classification, or a single-predicted-value
regression (or something else)?
Best.
K. Frank