I have run your code, but I do not find any bug.
Give you some advice:

If you initialized hidden state to zero, no operation required.


If (h_0, c_0) is not provided, both  h_0  and  c_0  default to zero. 51


If you set batch_first=True, and the shape of inputs is (batch_size, len)(50, 20)




 Raj:

input is a tensor from the data loader which has a batch size of 50 and sequence length of 20.


So, the hidden[0] should be (2, 50, 256) instead of (2, 20, 256)

h_0  of shape (num_layers * num_directions, batch, hidden_size): tensor containing the initial hidden state for each element in the batch.


For NLLLoss function,


Input: (N, C) where C = number of classes, and Target: (N) where each value is 0≤targets[i]≤C−1
So, the code should be:

        # reshape to be batch_size first
        soft_out = soft_out.view(batch_size, -1, self.output_size)
        
        soft_out = soft_out[:, -1, :] # get last batch of labels

4.If you use the lstm to complete the classification task, you can use the finally state or self-attention based on my experience.