Solved by vdw in post #2 
              
              
                Short answer: You can use either. 
For a bit more understanding, first check the output of output and h_n 


output.shape = (seq_len, batch, num_directions * hidden_size), so you get a (context) vector for each item in your sequence
h_n.shape = (num_layers * num_directions, batch, hidden_size)

Now …