Your general workflow might work, but:

DataLoader workers with LSTM load data [data1, …, datan], and then these tensors would have the LSTM autograd states [LSTM1, …, LSTMn].

an LSTM expects the data input as well as the cell and hidden state. If you are not passing the latter ones, they will be initialized with their default values and this wouldn’t match “and thus the LSTM must be stateful”.
So how are you passing the cell and hidden state from the previous iteration to the current forward pass of the LSTM inside each worker?