ptrblck:

What happens, if you change the seed to another number?


Thank you so much for your reply.
I changed the seed from 1234 to 12345 and it is working just as fine.
Below is the loss at the beginning of the 2nd epoch, before, meaning before setting seed values, it was in the range of 5000 to 10000. Now, it is slightly below the loss for the first epoch, which is exactly what I expect it to be.
I guess setting any kind of seed number is important for data augmentation that uses random number generator. I am not quite sure why.