Solved by gchochla in post #7 
              
              
                Your issue is that you are taking all the time steps in the series of hidden states without any aggregation. You should aggregate these outputs in a way that produces a tensor of size torch.size([20, hidden_size]), as you are trying to get a torch.size([20, 30495]) tensor as a result through a Linea…