When I wrap nn.DataParallel around my models that contain LSTMs I keep getting runtime errors when packing my padded sequences because the sequence lengths list that I pass in is not automatically split across the batch size.
`
    autoencoder = nn.DataParallel(autoencoder, dim=0)

    ...

    # Method of autoencoder called in forward
    def encode(self, indices, lengths):
            embeddings = self.embedding(indices)
            # error happens during packing 
            packed_embeddings = pack_padded_sequence(input=embeddings,
                                             lengths=lengths,
                                             batch_first=True)

            # Encode
            packed_output, state = self.encoder(packed_embeddings)
            hidden, cell = state

`
For example, for batch size 64, the word index tensors I pass in are split across the batch dimension (32 each), but the sequence lengths list that I pass into pack_padded_sequence is still of length 64. I am using batch dimension as the first dimension (dimension 0).
I’ve seen several posts related to this issue, but haven’t found a definitive solution for this. Do I have to make the sequence lengths list a tensor? Or do I have to make the batch dimension the second dimension (dimension 1)? Any suggestions?
Related posts:

  
    
    
    Does RNN work with DataParallel wrapper? 
  
  
    If I wrap RNN with DataParallel, it seems like output is not consistent with the target size. For instance, if the batch size is 32 and 2 gpus are active then 16 instances per gpu are processed. However, these instances should be aggregated in the end to get the whole batch of 32 instances for  loss function. But when I use rnn, aggregation is not happening and model outputs only 16 instances which is conflicting with the target value size. 
I don’t know it makes sense ?
  



  
    
    
    Question about: Packed RNN with DataParallel 
  
  
    @apaszke @stephenrawls 
I have a naive question here, in order to make the above forward function work on GPU,   the whole model and input have to be moved to GPU by calling .cuda(), right? But since pack_padded_sequence needs a seq lengths param (which is typically a list object if on CPU), so do I have to convert this list param into a cuda array? 
I’m having issues when doing this conversion, like LongTensor(input_lengths).cuda() and pass it to pack_padded_sequence.
  



  
      github.com/pytorch/pytorch
  
  
    



Issue: improve parallel performance for RNN models 87


	
	opened by soumith
	on 2016-09-26
	
	
	closed by soumith
	on 
	


Requests from M'aurelio and Sumit Chopra.
RNN Example with:
 DataParallel Multi-GPU
 ModelParallel Multi-GPU and part CPU - part GPU
 PipelineParallel
 RNN +...


 	enhancement
 	medium priority