Update: I ditched the reinforce method and I’m now simply computing the loss as logprobs * (reward - baseline), averaged over batch size B, where logprobs is the sum of the log probabilities of the outputs of the pointer network. The reward is computed based on the actions, not the action indices, which are the outputs of torch.multinomial.