Answers.Body: <p>As it says in the <a href="https://pytorch.org/docs/master/generated/torch.nn.RNN.html" rel="nofollow noreferrer">documentation</a>, you can simply reverse the order of dimensions by providing the argument <code>batch_first=True</code> when constructing the RNN.
  Then, the dimensionality will be: <code>(batch, seq, feature)</code>, i.e. batch-size <em>times</em> sequence length <em>times</em> the dimension of your input (however dimensional that may be). Then, everything is gonna work as you are used to it.</p>
  <p>To answer the second part of your question, normally, each sequence in a batch is independent of the others (since they commonly get sampled at random). So, there is no direct dependence between any two inputs in a batch (except, of course, for the fact that they are commonly expected to stem from some underlying shared data generating process which you want to approximate by the RNN).</p>
  <p>And a hidden state is commonly maintained per batch element, i.e. there is one hidden state per batch element (i.e. per sequence).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-08T12:11:44.097Z
Answers.Text: annoyingly this can mess things up. I was doing attention type stuff where I was interested in hidden state of encoders, but later when I did linear layers I had to have batch first. I thought that solving this would be to use `batch_first = True` but the output dimension of hidden state was still time x batch x features, much to my annoyance.
Answers.UserDisplayName: David
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-08T13:12:54.117Z
Answers.Text: Thank you, that explains it. But what is the logic behind this change in order which, as @david points out, can be rather inconvenient?
Answers.UserDisplayName: rboz22
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-08T19:33:16.917Z
Answers.Text: @rboz22 imo they probably added the `batch_first` argument before people started doing stuff where you explicitly need the hidden state, originally this was just a 'behind the scenes' thing that you would not be particularly interested in. This is of course just a guess.
Answers.UserDisplayName: David
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-08T19:59:18.973Z
Answers.Text: Also, I think it might be motivated by how real-life datasets are set up. I could imagine that there are many sets where you have for each time point a set of measurements (i.e. your batch elements) and then want to have sequence first, consequently.
Answers.UserDisplayName: Daniel B.
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-08T00:04:28.017Z
Answers.LastActivityDate: 2020-07-08T00:04:28.017Z
Answers.UserDisplayName: Daniel B.
Body: <p>I am confused as to why the sequence length is the first dimension of the input tensor for an RNN, while the batch size is the first dimension for any other kind of network (linear, CNN, etc.).</p>
  <p>This makes me think that I haven't fully grasped the concept of RNN batches. Is each independent batch a different sequence? And is the same hidden state across batches? Is the hidden state maintained between timesteps for a given sequence (for vanilla/truncated BPTT)?</p>
  
Tags: <neural-networks><recurrent-neural-networks><pytorch>
Title: In PyTorch, why does the sequence length need to be provided as the first dimension of the input tensor for an RNN?
UserDisplayName: rboz22
ThreadText: # In PyTorch, why does the sequence length need to be provided as the first dimension of the input tensor for an RNN?
  
  (Asked by: username_0 on 2020-07-07)
  
  I am confused as to why the sequence length is the first dimension of the input tensor for an RNN, while the batch size is the first dimension for any other kind of network (linear, CNN, etc.).
  
  This makes me think that I haven't fully grasped the concept of RNN batches. Is each independent batch a different sequence? And is the same hidden state across batches? Is the hidden state maintained between timesteps for a given sequence (for vanilla/truncated BPTT)?
  
  **Tags:** neural-networks, recurrent-neural-networks, pytorch
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-07-08. Score: 1
  
  As it says in the [documentation](https://pytorch.org/docs/master/generated/torch.nn.RNN.html), you can simply reverse the order of dimensions by providing the argument `batch_first=True` when constructing the RNN.
  Then, the dimensionality will be: `(batch, seq, feature)`, i.e. batch-size *times* sequence length *times* the dimension of your input (however dimensional that may be). Then, everything is gonna work as you are used to it.
  
  To answer the second part of your question, normally, each sequence in a batch is independent of the others (since they commonly get sampled at random). So, there is no direct dependence between any two inputs in a batch (except, of course, for the fact that they are commonly expected to stem from some underlying shared data generating process which you want to approximate by the RNN).
  
  And a hidden state is commonly maintained per batch element, i.e. there is one hidden state per batch element (i.e. per sequence).
  
  > **Comments:**
  > **username_2**: annoyingly this can mess things up. I was doing attention type stuff where I was interested in hidden state of encoders, but later when I did linear layers I had to have batch first. I thought that solving this would be to use `batch_first = True` but the output dimension of hidden state was still time x batch x features, much to my annoyance.  
  > **username_0**: Thank you, that explains it. But what is the logic behind this change in order which, as @david points out, can be rather inconvenient?  
  > **username_2**: @rboz22 imo they probably added the `batch_first` argument before people started doing stuff where you explicitly need the hidden state, originally this was just a 'behind the scenes' thing that you would not be particularly interested in. This is of course just a guess.  
  > **username_1**: Also, I think it might be motivated by how real-life datasets are set up. I could imagine that there are many sets where you have for each time point a set of measurements (i.e. your batch elements) and then want to have sequence first, consequently.  
  
  ---
  
  