In A3C, the gradients of main thread parameters are copied from training threads.
But the gradients of pytorch nn.module cannot be reassigned?
So how to implement this procedure?
BTW. is there a function like tf.gather_nd() in pytorch?
to extract values in specific indexes of the tensor (multidimensional).
Cause right now, the torch.indes_select can only select from one dimension.