I’ll pass on taking a cheap shot at other posts. 
But so here is the story beyond checklists:
Multiple workers help if you have CPU processing to do that can run on several cores or a situation where you don’t need much processing but have latency (e.g. reading from disk).
So if you have a TensorDataset with GPU tensors and no transforms, you would not not use workers.
One thing might help is if you replace loss.item() with loss.detach(). This would get you a list of GPU tensors (use stack(…).tolist() at the end to get the list of numbers back). Given that your computation is quite small, the CPU having to wait for the GPU to finish and transfer the loss might have an impact. (My GPUs are busy, so I couldn’t try…)
Best regards
Thomas