IIUC paths are treated as independent sequences, so you can treat their dimension as an additional batch dimension. I.e. map (batch_idx, path_idx) -> seq_idx, create a compact tensor of sequences, run a RNN, map results back and aggregate by batch_idx.