Did you run your cuda settings in one line like this:
CUDA_VISIBLE_DEVICES=1 python myscript.py

Could it be that you typed it before calling your python script, so that it has no effect and your models are still pushed on GPU 2, 3 and 4. This would explain the same memory amount of these GPUs.
However, if you really want to use only GPUs 1, 2 and 3, place the setting in one line and use
netG = torch.nn.DataParallel(netG, [0, 1, 2])
netD = torch.nn.DataParallel(netD, [0, 1, 2])

Could you check in which line the error was thrown and paste this code snippet?
You have to make sure, that all operations are executed on the same GPU id.
For example if your generator and discriminator are placed on different GPUs, you have to push the Variables to the according GPU:
netG = netG.cuda(0)
netD = netD.cuda(1)

outG = netG(Variable(data.cuda(0)))
outG = outG.cuda(1) # push to GPU 1

outD = netD(outG)

etc.
I hope it helps!