Selection_017.png603×600 79.9 KB

The microarchitecture of NVIDIA DGX-1 & Titan X is pascal only. I thought the difference between training on single GPU & Multiple GPUs(I’m guessing he trained on multiple gpu cores of DGX-1) is BN synchronization. BN sync leads to decrease the peformance but convergence do occur. Exp