zippeurfou:

#reload the initial dict so the hidden weight are not influenced by sentence_1 self.lstm.load_state_dict(state_dict)


I don’t think you actually need that (but you are missing a self in your forward signature definition).



 zippeurfou:

guess I can just join them on the feature dimension (seq x batch x feature*2) and then just split the output into two.


No, you join them in the batch dimension, or rather use packed sequences for that:
lstm = torch.nn.LSTM(300, 100)
sentence_1 = torch.randn(15,300)
sentence_2 = torch.randn(16,300)
def lstm_twice_1(sentence_1, sentence_2):
    if len(sentence_2) > len(sentence_1):  # order them for pack_sequence, doesn't matter as it's symmetric after the LSTM
      sentences = torch.nn.utils.rnn.pack_sequence([sentence_2, sentence_1])
    else:
      sentences = torch.nn.utils.rnn.pack_sequence([sentence_1, sentence_2])

    out, (h_n, c_n) = lstm(sentences)
    dist = torch.dist(h_n[0,0], h_n[0,1])
    return dist
    
def lstm_twice_2(sentence_1, sentence_2):
    out1, (h_n_1, c_n_1) = lstm(sentence_1.unsqueeze(1))
    out2, (h_n_2, c_n_2) = lstm(sentence_2.unsqueeze(1))
    dist = torch.dist(h_n_1[0, 0], h_n_2[0, 0])
    return dist

assert torch.allclose(lstm_twice_1(sentence_1, sentence_2), lstm_twice_2(sentence_1, sentence_2))
%timeit lstm_twice_1(sentence_1, sentence_2)
%timeit lstm_twice_2(sentence_1, sentence_2)

gives
100 loops, best of 3: 14.9 ms per loop
10 loops, best of 3: 24.9 ms per loop

This lazy IPython benchmark seems to suggest that

you can use the batch dimension to run two things through the same LSTM,
it’s a faster, too.

It becomes more tricky if you want batches of sentence pairs because you need to sort and keep track of the indices. But by then the relative speedup of variant 1 vs. variant 2 might become a lot smaller, too.
I’m not sure why @lugiavn thinks you can’t do that.
Best regards
Thomas