In your example you are already flattening the batch and temporal dimensions into the batch dimension before feeding the input to the model and would need to revert this operation before permuting the output and feeding it to the RNN.