Thank you very kindly. I indeed see my error.
Also, I noticed in the example that the saved_actions are drawn from a multinomial distribution (hence it is the stochastic variable). I am not all too familiar with gym’s API calls but it seems the rewards variable is the result of a deterministic env.step(...) call. Am I correct?
I have a similar code structure. I sample from a list of tensors using the multinomial function and then save the results from the sampling into a list of actions in my network model class.
What should the reward variable in my situation be? The output when I call forward on my stochastic variable by my lstm regressor? If I do that, it rightly raises a runtime error:
    raise RuntimeError("reinforce() can be only called on outputs "
RuntimeError: reinforce() can be only called on outputs of stochastic functions

So actions in your example is a stochastic variable, I agree. But you called action.reinforce on r and everything is honky dory. I guess my question is what is the rationale for doing this
    R = 0
    rewards = []
    for r in policy.rewards[::-1]:
        R = r + args.gamma * R
        rewards.insert(0, R)
    rewards = torch.Tensor(rewards)
    rewards = (rewards - rewards.mean()) / (rewards.std() + np.finfo(np.float32).eps)

and r in policy.rewards[::-1] is stochastic?