For me I use torch.cat to concatenate a list of torch tensors/variables (i.e a list of log prob or actions).
I’ve been able to do batch policy gradient this way very fast (faster than iterating through the list and summing up the loss).