Was just looking at the DCGAN tutorial (https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html 4) and noticed that .backward() is called twice:

Screen Shot 2020-03-01 at 9.30.38 PM686×730 94.5 KB

Wouldn’t it be more efficient to compute
errD = errD_fake + errD_real

first and then call
errD.backward()

so that backward has only to be called once (assuming it’s an expensive operation)?