I’m going to point you here: https://medium.com/mlreview/understanding-lstm-and-its-diagrams-37e2f46f1714 17 for the diagram.
The main idea is that we want the i, f, g, o gates to all be of size hidden_size. The math works out when the weight is (4*hidden_size, input_size) because the output of the matrix multiplication is split into 4 pieces.