Thanks for the response. I made sure to study more about PyTorch’s autograd, but I still have a question. Why exactly is having the gradient flow through the action log probability a bad thing in this case?