Thanks for the answer,
yes you are right, if you remove the activation_fn an RNN became almost a linear model.
I’m finishing the implementation, not sure how slow it would be. I hope it would be decent.