Example DCGAN: when updateing netD, noise can be volatile
<SCODE>noise.data.resize_(batch_size, nz, 1, 1)
noise.data.normal_(0, 1)
fake = netG(noise)
<ECODE> I think it would be better if we change it to: <SCODE>noise = Variable(torch.Tensor(batch_size, nz, 1, 1).normal_(0, 1),volatile = True)
fake = Variable(netG(noise).data)
<ECODE> because when training netD, we won’t need the buffer and gradient of netG. this may acclerate training step and reduce memory usage.[details=Summary]This text will be hidden[/details]
chenyuntc
You are correct, changing it like this is better. Even better, you can do: <SCODE>fake = netG(noise)
fake.detach()
<ECODE> If you are interested, please send a pull request to fix it, I will merge.
smth
detach according to the docs: What does it mean for the variable to be detached and how does it affect performance?
Veril
smth
Right, but it will also copy the volatile attribute of its predecessor, which means that you must either recast the variable anyway or not use volatile for the generator. So does detach serve a purpose here? I assume volatile in the generator is good, or why else have that flag option at all?
Veril
Because you don’t need to use volatile if you’re going to use detach later. It will be nearly equivalent, with volatile possibly being a bit faster and more memory efficient.
apaszke
<SCODE>-------------------------------------------------------------------
RuntimeError                      Traceback (most recent call last)
<ipython-input-25-5b95d5373ece> in <module>()
     19         fake_pic = netg(noise_).detach()
     20         output2 = netd(fake_pic)
---> 21         output2.backward(mone) #change for wgan
     22         D_x2 = output2.data.mean()
     23         optimizerD.step()

/usr/local/lib/python2.7/dist-packages/torch/autograd/variable.pyc in backward(self, gradient, retain_variables)
    149         """
    150         if self.volatile:
--> 151             raise RuntimeError('calling backward on a volatile variable')
    152         if gradient is None and self.requires_grad:
    153             if self.data.numel() != 1:

RuntimeError: calling backward on a volatile variable
<ECODE> <SCODE>> /usr/local/lib/python2.7/dist-packages/torch/autograd/variable.py(151)backward()
    149         """
    150         if self.volatile:
--> 151             raise RuntimeError('calling backward on a volatile variable')
    152         if gradient is None and self.requires_grad:
    153             if self.data.numel() != 1:

ipdb> u
> <ipython-input-27-5b95d5373ece>(21)<module>()
     19         fake_pic = netg(noise_).detach()
     20         output2 = netd(fake_pic)
---> 21         output2.backward(mone) #for wgan
     22         D_x2 = output2.data.mean()
     23         optimizerD.step()

ipdb> output2.volatile
True
ipdb> fake_pic.volatile
True
<ECODE>
chenyuntc
chenyuntc
Yes, volatile will be propagated even if you use detach(). I meant that you could remove the volatile flag and use detach with Variables that don’t require grad.
apaszke