Hi, firstly, thanks a lot of the reply. I am trying to build a classification model to classify certain sentence vectors as 1 and others as 0. So, I am passing sentence vectors, each of shape 1x512 to the model (Not sure if it’s relevant, but the shape of embedding matrix that I used was 579964x256). I am initially using 200 sentence vectors of shape 1x512 each to test the model to make sure that everything is working well before I start the actual training. But, the batch size is 1, so at each forward/backward pass, the input is one sentence vector of size 1x512. I have posted a sample sentence vector above in the post.
At each layer of the forward function, here’s how the shape changes:
Initial input: 1x512
After embedding layer: 1x512x512
After LSTM layer: 1x512x128
After first dropout layer: 1x512x128
After first linear layer: 1x512x64
After second dropout layer: 1x512x64
After second linear later: 1x512x1
After sigmoid layer: 1x512x1
After performing ‘view(1,-1)’ on the output from sigmoid layer: 1x512
Now, as I mentioned earlier, I am trying to classify it as 1/0 and I was a bit perplexed at the shape of sigmoid’s output being 1x512x1. Then, I referred to few other posts on pytorch forums and I found changing the output using view(1,-1), and it was posted by you, @ptrblck. So, thanks for that! Anyway, now after modifying the final output from Sigmoid using view(), I am still left with 1x512 shaped vector. So, I was wondering on what to do. I saw your answer recommending to use the last time step of the sentence vector (so, I am assuming that this would be index 511 in the final output array) or to calculate mean of all the 512 numbers in the vector. Is one of these inherently better than the other and are there any other corrections in other parts of the code?