Here， I have dumped all the tensorflow model weights into pytorch model.
I use a simple input in test and find that the forward pass and the backward pass are all the same.
In my case, I use resnet110 as the backbone. The dataset is cifar10.
The only difference is that I use cifar10 bin version in TF and cifar10 python version in pytorch.
Except this, the two models use the same initial weight, the same optimizer (SGD with momentum 0.9), the same batch normalization layer with momentum=0.99 and epsilon=1e-3, the same lr schedule and weight decay=2e-4.
But the results in pytorch are still worse than that in TF.
I really don’t know why.