I was going through the Chatbot tutorial 42 and saw the following code:
    def forward(self, input_seq, input_lengths, hidden=None):
        '''
        input_seq : vector of indices of the words (max_length, batch_size)

        Computation Graph:

            Convert word indexes to embeddings.
            Pack padded batch of sequences for RNN module.
            Forward pass through GRU.
            Unpack padding.
            Sum bidirectional GRU outputs.
            Return output and final hidden state.
        '''
        # Convert word indexes (10,64) to embeddings (10, 64, 500)
        embedded = self.embedding(input_seq) # (10,64,500) = (max_len, batch_size, embedding_dim=hidden_dim)
        # Pack padded batch of sequences for RNN module
        packed = nn.utils.rnn.pack_padded_sequence(embedded, input_lengths)
        # Forward pass through GRU
        outputs, hidden = self.gru(packed, hidden)
        # Unpack padding
        outputs, _ = nn.utils.rnn.pad_packed_sequence(outputs)
        # Sum bidirectional GRU outputs
        outputs = outputs[:, :, :self.hidden_size] + outputs[:, : ,self.hidden_size:]
        # Return output and final hidden state
        return outputs, hidden

I don’t understand what the purpose of packing is. I assume is that I first don’t understand the problem these functions are trying to solve so I am going to state what I think it is and hopefully this can help people understand whats going and correct it:

Problem statement: process batches of different/variable length in Pytorch efficiently (rather than manually)

This is my understanding of what is going on and how the functions solve it: We have to process batches of DIFFERENT lengths but we don’t want the padding to be processed by our RNN modules. So we need to tell our RNN what the padding is to process it properly. This is better than processing each sequence in the batch ourselves in pytorch by looping through it. My guess is because pytorch might to some optimization (with GPUs especially) than our code (in Python) sequentially going through each sequence in the loop. Adding parallelize ourselves seems silly.
So we pack the (zero) padded sequence and the packing tells pytorch how to have each sequence when the RNN model (say a GRU or LSTM) receives the batch so that it doesn’t process the meaningless padding (since the padding is only there so that things are tensors, since we can’t have “tensors of each row having a different length”)
Is this correct? Is this why we need padding?

Crossposted:
https://www.quora.com/unanswered/Why-do-we-need-to-pack-padded-batches-of-sequences-in-Pytorch 98

Resources I’ve read to understand this issue:
https://www.quora.com/What-does-packing-a-sequence-mean-in-Pytorch 64
https://www.quora.com/unanswered/Why-do-we-need-to-pad-sequences-by-ourselves-in-Pytorch-when-we-supply-the-sequence-lengths-Why-isn-t-pad_packed_sequence-sufficient 45

  
    
    
    Understanding pack_padded_sequence and pad_packed_sequence 
  
  
    Hi, 
I have a problem understanding these 2 utilities. Not able to figure out what it does. 
For eg. I was trying to replicate this with example from Simple working example how to use packing for variable-length sequence inputs for rnn 
I have followed the pytorch documentation and coded with batch First 

import torch
import torch.nn as nn
from torch.autograd import Variable

batch_size = 3
max_length = 3
hidden_size = 2
n_layers =1
num_input_features = 1
input_tensor = torch.zeros(batch_size,m…
  



  
    
    
    Why isn't pad_packed_sequence sufficient ? Why do we need to pad by ourselves when we supply the sequence lengths? nlp
  
  
    Why do we need to pad the input for variable length sequences for lstm when there is a pack_padded_sequence function that essentially tells the lstm to ignore the padded portion? Why isn’t the pack_padded_sequence function with the sequence lengths sufficient for training via mini batches? Why do we need to pre-pad the input ?
  


https://www.quora.com/unanswered/Why-do-sequences-need-to-get-sorted-for-packing 22

  
    
    
    Why do sequences need to get sorted for packing? 
  
  
    In order to make an LSTM deal with variable length sequences in a batch, I have to use pack_padded_sequence first. But why do the sequences for a batch have to be sorted in decreasing order of length? In order to match each of the sequences with the correct training signal, I then have to unsort the output of the unpacked sequences again. I do not understand why this reordering is necessary, it makes everything a whole lot more complex in the code. 
What I also do not understand is why there is …