Usually we are seeing different convergence behavior between frameworks for a variety of reasons, such as a mismatch in the model architecture, another initialization of the parameters, different usage of schedulers and optimizers, different pre- or post-processing etc.
The memory format wasn’t on in this list so far, so do you observe that the model converges in one memory format and fails in the other?