I’m creating a sequence to sequence model based on an auto-encoder to map a sequence of words to another sequence of words. The data I am using is sizable so batch training is essential. Now, defining the encoder seems to be straightforward. But I was wondering what should be the input of the decoder since at each time step t the decoder receives a hidden vector and its own output from t-1 as input.
This can be seen in the following figure from Sean Robertson 2’s pytorch tutorial 13’s and in Sutskever et al (2014).

I am not sure how to pass the output of the decoder to itself as an input of the next time step when training with batch.
Below is how I define an encoder and decoder, but the input of the decoder is missing.
class seq2seq(nn.Module):

def __init__(self, ):
    super(seq2seq, self).__init__()

    self.embed = nn.Embedding(embedding_input_size, embedding_hidden_size)

    self.encoder = nn.GRU(input_size=embedding_hidden_size,
                          hidden_size=encoder_hidden_size,
                          num_layers=1)

    self.decoder = nn.GRU(input_size=encoder_hidden_size,
                          hidden_size=decoder_hidden_size,
                          num_layers=1)

    # sos_vec is the one hot vector representing the start of sequence 
    decoder_first_inp = Variable(torch.LongTensor(sos_vec))


def forward(self, input_btch, hidden_btch):

    encO, encH = self.encoder(input_btch, hidden_btch)

    decO, decH = self.decoder(???, encH)