Do you really read something about the difference between these two implementations?
In TF, the operation is 1/2 * sum(weight * weight). The grad is the same.
Since TF use static computation graph, the Jacobian is straight foward.
I will try to train the model without weight decay.
But in my model, the final evaluation result relies on the classification accuracy.
If the weight decay implementation is the curprit, this is a bad thing. It means that I can’t use pytorch to train the model.