abhinav_choudhury:

So at the end I used Adam optimizer.


Did you also compare the TF model using Adam?
Before diving into the code it would be good to have a valid baseline to compare against and currently it seems that the scripts differ at least in the optimizer.