Solved by ZetilenZoe in post #3 
              
              
                You can have a look at how LSTMCell is implemented in the file _functions/rnn.py 
You can see how the gates are computed in the line 32. 
gates = F.linear(input, w_ih, b_ih) + F.linear(hx, w_hh, b_hh)

Here, weights are of shape (4*hidden_size x input_size), (4*hidden_size x hidden_size) and biases …