Hey all,
I would like to ask how the hidden states produced by a Bidirectional RNN are concatenated.
If I’m not mistaken, the output parameter of a PyTorch RNN is of shape (N, T, 2*H) given, that the ‘batch_first’ and ‘bidirectional’ parameters have been set to True [N: number of examples, T: number of time steps, H: cell size].
Are the two 3-D tensors concatenated on the last axis? Or is each pair of tensors computed on a given time step “put” next to each other?
I would like to merge the hidden states of the Bidirectional RNN (to produce an output of shape (N, T, H)). If I’m correct there is a difference between torch.split() and torch.chunk() functions.
Thank you for your help in advance!
Bence