The  code of dcgan is
if opt.cuda:
    netD.cuda()
    netG.cuda()
    criterion.cuda()
    input, label = input.cuda(), label.cuda()
    noise, fixed_noise = noise.cuda(), fixed_noise.cuda()

So, is the criterion use one gpu ? If it just use one gpu , how about the backward? Is it use one GPU too?