Hi,
You should not call the .forward() function of Modules directly but just call them like: actions = PolicyNet(state).
If your PolicyNet’s params have requires_grad=True, then actions will also have requires_grad=True and it will propagate all the way to the loss.
So you want to make sure that the PolicyNet params do require grads and that you don’t .detach() (or .data) anything during the loss computation.