I have trained the TF model for more than fifty times.
In all cases, the results are good enough.
I also trained the pytorch model for nearly twenty times. All results are not so good.
Until now, the only thing I have not checked is the optimization procedure.
But I have checked the SGD code of pytorch, the weight decay is implemented sightly different with TF.
In TF, the L2 loss is added into the total loss.
In pytorch, the L2 loss is directly implemented in the grad manipulation.
Theoretically, they are the same.
But I don’t know whether these two implementation have difference for the sake of precision or approximation.