Thanks for your response. I actually figured out the reason for the undeterministic behaviour. It was the dataset, some indices I was creating to feed a char-embedding layer were being constructed non-deterministically.