@D_Liebman I was also having trouble understanding the dimensions of the hidden state when I moved my encoder from one direction to bi-directional. The exact problem you had with initHidden. I was uncertain if I should have result = Variable(torch.zeros(1, 1, self.hidden_size)) or result = Variable(torch.zeros(2, 1, self.hidden_size)). When I tried torch.zeros(2,1,self.hidden_size), which was what I thought I was correct, I got an error that it can’t convert more than a single value to a python scalar, and so I went back. Not quite sure, sorry, but I’m here with you in being confused lol. Also, I was using it for self.hidden = self.initHidden() and storing the hidden state in the encoder class, so I think I do need that function.
I have another question about bidirectional seq2seq. Can you replace the GRU with an LSTM? LSTM works fine with attention, right? It seems the LSTM’s get very nice results in papers, and I don’t see a reason to not use an LSTM (but I’m having trouble implementing it).
I’ll keep up with this thread and help out if I figure stuff out.