Solved by a_d in post #4 
              
              
                Hello 
According to your code,  you are not passing  the vanilla input but rather an attentive input into your input gate which is of shape batch, hidden_size. The W_i which maps from input to hidden is of input, hidden, and therein lies the problem. 
Also, based on a cursory reading of the paper, t…