@ptrblck thanks for your reply.  T4 is slow , i guess it is because the batch_size is too small. I reproduce the code in https://pytorch.org/tutorials/recipes/recipes/amp_recipe.html 8 .
The conclusion is mixed precision is slow in Tesla k80, and faster in T4.
I wanna to speed up the interference speed, and i only have k80 dev environment and t4 product environment. it is ok to use Tesla k80 to train and use T4 to interference in amp ?