Solved by vdw in post #4 
              
              
                Argh, so sorry, my bad! The line 
last_h = h[0].view(self.n_layers, 2, batch_size, self.hidden_dim)

was supposed to be 
last_h = h[0].view(self.n_layers, 2, batch_size, self.hidden_dim)[-1]

The [-1] gets the last of your 3 layers. Then the shape of l_last will be (2, batch_size, hidden_dim), with …