Assume (x,len) is the input to your packed RNN, (where x is a (batch_size, max_len, d_embed) tensor, len is a list of length batch_size), then you can use (x.cuda(), torch.LongTensor(len).cuda()) to distribute them on GPUs, and then in the model (which have already distributed on GPUs), use (x,list(len)) as the input. It’s a straightforward and inefficient method… Am I clear ?