If you must use the same machine whilst training, either improve the hardware or sacrifice the training speed.
As for the latter, monitor your resources with htop. If you lack memory, either turn down DataLoader's num_workers or batch_size. If you lack CPU cores, turn down DataLoader's num_workers only.
You can also monitor disk IO with atop; if you have a single disk and it’s 100% busy, use h5py to zip your data in prior. Or, get a new disk and seperate your OS from your data.