Instead of replacing the grad_output, shouldn’t you scale grad_output by the reward.
The output is softmax https://github.com/pytorch/examples/blob/master/reinforcement_learning/reinforce.py#L43 95 which I understand, but there is no cross-entropy loss in the case. Am I missing something?