Currently the code assumes the output of the embedding layer is [seq_len=1, batch_size=1, features=-1], since it passes each input one by one.
If you want to use nn.DataParallel, you would need to provide the input as a batch, which means you might have to add padding to it or cut it to a certain length.
This batch will be split among the batch dimension (dim0) and passed to each model.
The current forward method of your models assume a single word as its input. You would have to adapt this code, so that the input to nn.Embedding would be [batch_size, indices] and a tensor of shape [seq_len, batch_size, features] can be passed to the nn.GRU.