Initialization did not solve the problem either.
1- What is in the kreas model that is not in the pytorch model?
2- Did I write the pytorch model correctly according to the keras model?
3- What is your advice to solve this problem?
Loss decreases from 0.1 to 0.01 in the keras model, but stops at 0.08 in the pytorch model (most of the time).