I have an RCC which outputs tensorrs of size
516x24 that I’d like to get to size
4x6. I have a single linear layer that reduces it to
516x6, but I don’t know how to reduce the 1st dim of
516. Any suggestions? Thanks!
I have an RCC which outputs tensorrs of size
The first dimension is just the batch size. So you can just lower your batch size if you want to get the output to be smaller.
My
batch_size is
4. Input
X is
(batch_size, 129, 64). RNN outputs
(batch_size, 129, 516). Then I pass the output through a
.contiguous.view(-1, 516), which gives me a size of
(4, 516), which I pass through a linear layer to get size of
(516, 6). However, I need it to be
(4, 6) at the end.
Can you show me your code?
I think I figured it out, by changing the input size of my linear layer to be large enough. I don’t know if this is the best approach though. Here’s my model code:
def __init__(self, input_size, output_size, hidden_dim, n_layers): super(Net, self).__init__() self.input_size = input_size self.output_size = output_size self.hidden_dim = hidden_dim self.n_layers = n_layers self.rnn = torch.nn.RNN(self.input_size, self.hidden_dim, self.n_layers, batch_first=True) # 64 is the size of my middle (2nd) dimension of X self.fc = torch.nn.Linear(64 * hidden_dim, self.output_size) def forward(self, X): batch_size = X.size(0) # 4 hidden = self.init_hidden(batch_size) out, hidden = self.rnn(X, hidden) out = out.contiguous().view(batch_size, -1) out = self.fc(out) return out, hidden def init_hidden(self, batch_size): return torch.zeros(self.n_layers, batch_size, self.hidden_dim)```
That looks good. One question however, why are you returning the hidden layer if you just reinitialize it every forward pass? Doesn’t that defeat the purpose of returning it.
I was following the tutorial below, which does that. I think it’s just initialising the very first hidden state for each batch, isn’t it? I don’t use the return value though. Does it serve any use outside of training the model?
No it doesn’t look like it does. It does not matter much though I am glad you fixed the issue.