You can have a look at how LSTMCell is implemented in the file _functions/rnn.py 6
You can see how the gates are computed in the line 32.
gates = F.linear(input, w_ih, b_ih) + F.linear(hx, w_hh, b_hh)

Here, weights are of shape (4*hidden_size x input_size), (4*hidden_size x hidden_size) and biases are of shape (4*hidden_size). The result of linear function will be of shape (4*hidden_size). Then, there comes the splitting part in line 34.
ingate, forgetgate, cellgate, outgate = gates.chunk(4, 1)

Then each gate will be of shape (hidden_size).