Finally figured it out.
return F.log_softmax(self.linear(F.tanh(output[:,-1,:]))) . In this snippet F.tanh was not necessary since it happens in  each of the  lstm cell implementation by default. I think that is where I was going wrong. Removed it and I am able to get equivalent results as in keras. Do correct me if I am wrong.
Thank you very much for your help.