Seems that someone else is facing some similar problem.

  
    
    
    Suboptimal convergence when compared with TensorFlow model vision
  
  
    I ported a simple model (using dilated convolutions) from TensorFlow (written in Keras) to pytorch (last stable version) and the convergence is very different on pytorch, leading to results that are good but not even close of the results I got with TensorFlow. So I wonder if there are differences on optimizers in Pytorch, what I already checked is: 

Same parameters for optimizer (Adam)
Same loss function
Same initialization
Same learning rate
Same architecture
Same amount of parameters
Same dat…