It seems to me that you are working with pairs of 3D tensors from the initHidden but you covert your embedded input into a 4D tensor. I may be mistaken but this seems the most obvious at first glance.