Recently, I found pack_sequence, pack_padded_sequence, and pad_packed_sequence for RNN modules. But I am not sure when these functions are useful.
Q1. To use pack_padded_sequence,  sorting tensors by length is needed for every mini-batch. This is cost. On the other hand, I heard pack_padded_sequence skip calculation of padded elements. Compared them, skipping calculation is much beneficial? The below link is an example of pack_padded_sequence.

  
      gist.github.com
  
  
    https://gist.github.com/Tushar-N/dfca335e370a2bc3bc79876e6270099e 74
pad_packed_demo.py
import torch
import torch.nn as nn
from torch.autograd import Variable
from torch.nn.utils.rnn import pack_padded_sequence, pad_packed_sequence
import torch.nn.functional as F
import numpy as np
import itertools

def flatten(l):
	return list(itertools.chain.from_iterable(l))
This file has been truncated. show original




  
  
    
    
  
  


Q2.  Although pack_sequence looks similar to pack_sequence, pack_sequence doesn’t need to use padded data. But what kind of situations can we use this function?
Thank you!