I think calling backward once on errD_real + errD_fake should be the same as calling it twice separately. The problem mainly attributes to reusing the same label tensor for real and fake data (https://github.com/pytorch/examples/blob/master/dcgan/main.py 200, L219, L232). If creating two label tensors, the problem should be resolved. Nevertheless, calling backward twice is recommended as it could save some GPU memory.