https://pytorch.org/tutorials/intermediate/reinforcement_q_learning.html 39
while the comments in the tutorial specify that autograd is used, it is never explicitly declared (that I can see). In supervised learning, the inputs are usually set as input_data = Variable(input_data) and then out = net.forward(data). However, here, Variable is never used. I do see that the loss tensor contains a gradient - but I am not sure where this came from.
Another observation, if I set
state_action_values = Variable(state_action_values,requires_grad=True)
then the code will not run - throwing an error on:
for param in policy_net.parameters():
param.grad.data.clamp_(-1, 1)
saying that ‘NoneType’ has no attribute data (where as clearly before adding the Variable code it did…)
Any ideas? Why is Variable not necessary here?