Yes, maybe 
Somewhat related question: how to get the output of the encoder and decoder in a pytorch idiomatic way, and handling teacher forcing etc?
soemthing like? :
def forward(self, encoder_input, decoder_input, state):
    if encoder_input is not None:
        enc_input_emb = self.embed_seq2seq(encoder_input)
        enc_out, state = self.lstm_enc(enc_input_emb, state)
    if decoder_input is not None:
        decoder_input_emb = self.embed_seq2seq(decoder_input)
        dec_out, state = self.lstm_dec(decoder_input_emb, state)

    embedding_size = self.embed_seq2seq.size()[1]
    batch_size = decoder_input.size()[1]
    seq_len = decoder_input.size()[0]
    dec_out_unemb = dec_out.view(-1, embedding_size) @ self.embed_seq2seq.weight.transpose(0, 1)
    dec_out_unemb = dec_out_unemb.view(seq_len, batch_size, -1)
    return enc_out, dec_out, dec_out_unemb, state

?