Hi, thanks for the help. I transposed the matrix to get a 128 x 128 as result, but got into another problem that it can’t be multiplied in the input gate by the weight because the weight is shape 32 x 32.

PyTorch856×583 32.6 KB

Note: I’ve tried with a 32x32 result, and it can do the multiplication, but the input and hidden state sum in the input gate doesn’t work because of the shape as well.

PyTorch21037×42 2.4 KB