MinsooKim:

Currently, I am trying to construct this architecture in one single nn.Module like the second of the image. Mini-batch consists of samples with various classes.
After FC layer, split batch for each class.
Then, get into each decoder, then concatenate all output of decoders as final output.
Using this final output, calculate loss and back-propagation.
Is it a correct way to train them?


This approach looks correct for training.
However, how would this method work for inference (or validation and testing), if no labels are available?