I would recommend to profile the code directly by synchronizing the code via torch.cuda.synchronize() before starting and stopping the timer and calculate the average time. Currently you are also profiling the startup time etc.



 Hu_Penglong:

Is it because，Tesla 80k do not have Tensor Core?


Yes, this would explain the slowdown on the K80 due to the added overhead in amp.



 Hu_Penglong:

I change a Tesla T4, get the same result.


I’ll try to grab a node with a T4 to reproduce it.