Hm okay thank you. Why is a forbidden task considered as a “higher reward”? Shouldn’t it be the negative or inverse like
grad_logprobs *  (returns - forbidden)

Wouldn’t that emphasize to do the forbidden task more often?
Or am I understanding something completely wrong here?