I think they are not concatenated in the literal sense, this is just to simplify notation. According to the docs 2 the forget gate is computed like so:
sigmoid(bfi + sum(U_t * X_t) + sum(W_t * h_t-1 +bfh)
With U_t being the input weights at time t and W_t being the recurrent weights  at time t. So there are 2 weight matrices that are used in each cell. I am definetely not an expert so maybe wait for a more accurate response.
P.S: Someone plz tell me how to use latex in here