No, increasing num_workers in the DataLoader would use multiprocessing to load the data from the Dataset and would not avoid an out of memory on the GPU.
To solve the latter you would have to reduce the memory usage by e.g. reducing the batch size or by using e.g. torch.utils.checkpoint.