Hi Tùng!



 tungdop2:


When i should split my output and when i should combine them like your way?



It really doesn’t matter – it is a purely stylistic choice.  The two versions
are completely* equivalent.  In my version there is a single Linear
whose weight has shape:
[n_classes_layout + n_classes_style, last_channel],
while your version has two Linears with weights with shapes:
[n_classes_layout, last_channel],
and
[n_classes_layout + n_classes_style, last_channel].
But it’s the exact same matrix multiplying the exact same input tensor,
but in your version the matrix and computation are just broken up into
two pieces.
*)  The two versions do actually differ in that your version has two
sets of random Dropout choices, but I think that this difference is
immaterial.



I think my model will be better if i know the relation between output labels



When your labels have structure, it is sometimes be better to build
that structure into the architecture of your model so that your model
doesn’t have to “learn” the structure.  But it is also sometimes better
not to build the structure into your model, and, instead, rely on your
model to “learn” the structure.  It depends on your use case.
By way of example, suppose you have a multi-label, ten-class
problem, and you happen to know that you will never have more
than eight labels active at the same time.  That is, your multi-label
label could have any number from zero to eight of the labels active
at the same time, but never nine or ten.  You would be better off
not trying to build that structure into your model – just train your
model well, and it  won’t (very often) predict more than eight labels
being active at the same time.
On the other hand, suppose you have four classes, A, B, C, and D,
and you have a multi-label problem where you know that exactly
two of the labels will be active at the same time.  You may well be
better off recasting your problem as a single-label, six-class problem
where the six classes are AB, AC, AD, BC, BD, and CD.  I could
well imagine this version would train more easily because it doesn’t
have to “learn” that multi-labels such as B or ACD or no label at
all never occur.



And why i shounldn’t  use Sigmoid layer?



Sigmoid followed by BCELoss is mathematically equivalent to
BCEWithLogitsLoss, but numerically it is less stable.  Internally
BCEWithLogitsLoss combines the Sigmoid with a log() function
into a log_sigmoid() implementation that uses the log-sum-exp trick
for improved numerical stability.
Best.
K. Frank