After forcing the to inputs to be exactly the same, and in the exact same order sequentially (disabled shuffling for extra measure), I finally started getting same initial loss evolution, though they slightly differ towards the end and give slightly different results, but at least it’s much better than before. Thanks a lot!