You might not have enough shared memory, so you could try to increase it on your system (or docker, if you are using it).
I would also recommend to update to the latest stable PyTorch version (1.5) just in case you are hitting an older bug.
If you are using multiple workers in your DataLoader, you could also try to set num_workers=0 for the sake of debugging.