chaslie:

should they be converted to cuda in the custom dataset?


No, the usual workflow would be to push the data to the GPU inside the DataLoader loop.
Could you come up with an executable code snippet using random input tensors, which would show the increase in GPU memory usage after setting the num_workers to a value larger than 0, so that we could reproduce and debug it?