Just to rule out one source of architecture differences: is mixed precision (e.g., fp16 training) or some other form of reduced-precision being used? More variability across architectures would be expected with reduced precision.