@Milas Thank you for the update. I doubt that heterogeneous GPUs is the issue… I would expect the numerical behavior to be largely identical across architectures. If you also mix library versions (e.g. cuDNN) this is more likely though.
Any more information sure is appreciated!