@quanguet
Thank you for your explanation. Regards to your comment about out[:,:,-1], the full shape of out is fed to linear then logtis[:,:,-1] is extracted to the loss function. Will this still not be enough to learn anything?
 def forward(self, x):
        out, (hn, cn) = self.lstm(x)
        # out shape: [bs, seq, hidden_size]

        logits = self.linear(out)
        return logits[:,:,-1]

If so, what will be the best way to tackle this? as I only have one dimension of label. Will large dataset can solve the issue?
Thank you!