Yeah, I think what’s going on is in RL, we look at the state at one timestep (size: [65]).  The batch size could be greater than 1, but we still pass a single time_step of state to the model. So the model doesn’t look at [32, 65], but [65].  But nn.parallel expects more shape[0] to be greater than 1, like you say it splits in dim0.
Most examples I see have a Variable declared with something like [batch_size, num_features] but mine doesn’t so I’ll have to figure out whether or not I want to keep passing a single example (I suppose if I want multiple GPUs I do have to change it).
Thanks for the feedback