You might want e.g. to pass the hidden state from one sequence to another one. The tutorial gives you a nice introduction and shows how to handle the hidden state.
You might want to use a Dataset and DataLoader to create batches with the option of using multiple workers. Have a look at this tutorial 10 for more information.