A couple points about 2:


In your code, assuming your comments are correct, transforming to/from RNN input/output is a transpose on dim 0 and 1. View wouldn’t work in this case. I’d replace all those with transpose(0, 1) or just add batch_first = True to RNN constructor.


I played with log_softmax a bit and did some investigation. The default implicit dimension choice on 3d case may not be what you want. It’s something weird and will be deprecated soon. Next version will add a dim arg, which will be perfect for this use case. For now, I’d fix this by (including fix for 1)


        self.rnn = nn.RNN(embedding_dim, hidden_size, nlayers, dropout=0.5, batch_first = True)

......


        # -- encoder returns:
        # -- [batch_sz, seq_len, embed_dim]
        encoded = self.encoder(inputs) 
        _, seq_len, _ = encoded.size()

        # -- rnn returns:
        # -- output.size() = [seq_len, batch_sz, hidden_sz]
        # -- hidden.size() = [nlayers, batch_sz, hidden_sz]
        output, hidden = self.rnn(encoded, hidden)

        # -- decoder returns:
        # -- output.size() = [batch_sz, seq_len, vocab_size]
        dec_out = decoder(output)
        output = F.log_softmax(dec_out.view(batch_size * seq_len, -1).view(batch_sz, seq_len, self.hidden_size)


Finally, are you trying to do a seq2seq task where you don’t know the output seqlen before hand, such as translation? If that is the case enc-dec models where both encoder and decoder are recurrent are a better fit!