Hi @SimonW,
Thanks! for the reply.
The mean and std are outputs of a policy network (continuous actions), but that is not really important as it could also be a classification problem.
I see what you are saying about being trained. However, I am trying to manually train the weights, i.e. I am computing gradients w.r.t a specific set of weights as opposed loss.backwards() (which would only train layers registered as module parameters, if I understand correctly)
So as long as I am updating the weights, it should be ok?
Thanks