Expected speed-up is about 3x compared to P100, 2x compared to V100 fp32 (not 8x). To get those numbers on a single GPU, you’d have to use batch size of 128 or 256 per GPU. On a single GPU, expected time per iteration is ~180ms for a batch of 128. To get good multi-GPU results, don’t use nvcr images, you’d need pytorch compiled from current master and run examples from here https://github.com/csarofeen/examples/tree/dist_fp16 20, it uses distributed mode and nccl backend.