The one other thing you can consider is not loading the whole thing into memory at once and use the provided Dataset/DataLoader constructs to parallelize the loading from disk to make sure the decoding/preprocessing is not slowing down your training.