Oh sorry I didn’t notice that.
Thanks for your patient explain!
After add synchronize() before count the time.
I found It’s indeed the synchronizing step consume the time.
I just want make sure is it normal that the V100 is faster so much than P100 on fp16 inference?
After add synchronizing time, As the result, the speed of V100 is about 20x faster than P100.
If not, what may be the problem?