I tested the inference of the vgg16 network with the pytorch framework, and found that the first layer of network (convolutional layer) for the first inference took a very long time, probably dozens of times the total time of the second reasoning. I know this may be the first Once booted on the GPU, but I want to know what the hell are doing this time?
Machine Configuration：GTX2080Ti
key Code：
cnt = 0
for i in gpu_input_data:
y = net(i)
cnt += 1
time.sleep(0.02)
if cnt > 20:
break
The timeline measured with pytorch’s profile is as follows：
first time inference: 300ms, the average cost time the rest 19 times inference: 13ms