If you have a look at the source code 20 of LSTMCell, you can see how cell and hidden states are computed in the lines 41 and 42.
cy = (forgetgate * cx) + (ingate * cellgate)
hy = outgate * F.tanh(cy)

Since all operations are element-wise, all gates and states have the same dimension which turns out to be hidden_size.