I was looking at the definition of the loss for REINFORCE from https://rllab.readthedocs.io/en/latest/user/implement_algo_basic.html which had it defined as:

I was wondering why in the example implementation (https://github.com/pytorch/examples/blob/81f47e8ea49c74494d2aa8dc1c9c4ddc6c0eca73/reinforcement_learning/reinforce.py#L71 1) there is no division by the number of steps (T)?