Hi @guyrose3
Sorry for misunderstanding your question. 
Going by your updated example, you seem to be doing the right thing, i.e.
h_sum = h1_new + h2_new
out = fc(h_sum)

This of course means that you cannot use the num_layers keyword argument in nn.LSTM, but will need to make the call individually for each layer. But for your use case, I guess this is as optimal as it can get at the Python level.