Trying to model different sized elements in a batch. Wondering if this kind of processing is valid:

Pack a padded sequence and feed through LSTM.
Unpack
Shift dimensions to simulate timedistributed.
Reshape back into original shape
Am I done with processing at this point? Part of me thinks I need to zero out all the items past the seq length for each element.

Does this make sense?
input_seq = early_rnn.pack_sequence(input_seq)
    # ----------------------------
    # RUN THROUGH SUBNET
    # ----------------------------
    # io_rnn_output (RNN output at each timestep) = [steps, batch_size, x_features]
    io_rnn_output, (io_h_n, io_c_n) = self.io_rnn(input_seq, self.io_rnn_hidden)
    io_rnn_output, seq_lengths = pad_packed_sequence(io_rnn_output)

    # dense output
    io_rnn_output = self.time_distributed(self.io_dense_1, io_rnn_output)
    io_rnn_output = self.time_distributed(F.relu, io_rnn_output)
    io_rnn_output = self.time_distributed(self.io_dense_2, io_rnn_output)
    io_rnn_output = self.time_distributed(F.sigmoid, io_rnn_output)    

    # what's next?? zero out what used to be the zeros in the packed sequence? or just flow this into the rest of the net?