Sometimes subtle differences in the definitions can make comparisons difficult, e.g. when a mean is taken in the loss function when your comparison uses a sum, you would need to compensate for that.
Also there are quite a few parameters to the optimizer that have differing defaults between implementations.
I’d probably try to start both with the same weights and see where it differs.
Best regards
Thomas