Why is there CUDA out of memory issue when you increase the num_workers? There shouldn’t be OOM error when num_workers is increased as it’s not related to GPU at all.
Can you check whether you are changing batch size or model input when increasing the num_workers.