Hi everyone,
I hope to trian the REINFORCE algorithm in a batch mode with batch size larger than 1. From this example: https://github.com/pytorch/examples/blob/master/reinforcement_learning/reinforce.py 69
I have a feeling that this code need to be modified:
for action, r in zip(model_Net.saved_actions, rewards):
    action.reinforce(r)
optimizer.zero_grad()
autograd.backward(model_Net.saved_actions, [None for _ in model_Net.saved_actions])

My question is how to pass the ‘r’ and ‘action’ in a batch mode in back-propagation ? It might related to reshape the ‘action’ values in a way to allow back propagation.
Right now I came up with an idea that is to compute ‘r’ and ‘action’ in batch mode in forward passes, but update the gradients sequentially (1 sample at a time) in back-propagations (e.g., run ‘finish_episode’ several times). But it’s obviously not optimal.
Thanks in advance,
Rein