Yes, your example is clear. I see, so because the module Parameters inside the network are automatically set to requires_grad, then everything that goes through it gets a grad.
I just modified the DQN to load in a pre-trained network. It looks like it was trained versions ago (and used Variable). Now - I run into the same problem as before



 Ranahanocka:

or param in policy_net.parameters():
param.grad.data.clamp_(-1, 1)
saying that ‘NoneType’ has no attribute data


I guess this is because of the fact that the stored model dictionary had a Variable in it right (their module defined Variable within the network itself)? This means that I can’t use older pre-trained networks on torch 0.4+?