You could wrap it into a DataLoader which would use multiple workers etc. once you use the PyTorch data classes.
In DistributedDataParallel you would use a DistributedSampler (probably not possible or necessary in your case as you are randomly generating the data). Besides that each process will create the Dataset or DataLoader. In the latter case each worker will create a copy of the passed Dataset.