I have a question related to this post so I thought I would follow up here rather than creating a new post. I’ve currently been doing the following to extract the output at the last (non-padded) timestep for an LSTM I’m using (note: batch_first is True here):
def encoder(self, input: torch.Tensor, lengths: torch.Tensor)
    # pack our zero-padded sequences
    packed_seq = nn.utils.rnn.pack_padded_sequence(input, lengths, batch_first=True)

    # pass input through lstm
    packed_output, _ = self.lstm(packed_seq)

    # unpack the output which is (batch x seq_len x num_directions * hidden_size)
    total_length = input.size(1)
    output, _ = torch.nn.utils.rnn.pad_packed_sequence(packed_output, batch_first=True, total_length=total_length)

    # grab the final output for each sequence in the batch
    seq_len_indices = [length - 1 for length in lengths]
    batch_indices = [i for i in range(self.batch_size)]
    final_output = output[batch_indices, seq_len_indices, :]

    return final_output

My final_output has the right dimensions (batch_size x hidden_size) and seems to get output from the right timesteps, but I’m wondering if I need to be using the torch operations people used above^ (e.g. the gather function with masks) in order for autograd to backpropogate the loss correctly? Will slicing the output tensor using regular lists mess up the backprop?