i think you didn’t understand my problem. activation functions mathematics we all know right. it doesn’t matter weather I use relu or tanh as activation function when I m using trained weights by Pytorch RNN module and that is giving fine results then why m self coded RNN is not giving similar results as pytroch module. and that is only when I use relu as a activation function , when I use tanh in pytorch module and match the ans with my self coded RNN both are matching that means there is no problem in my codes. i just want to know what type of relu exactly they used while building RNN.