Hello,
I am currently using pytorch (1.6.0) to build a graph convolutional network, but when trying my model in inference, I was faced with this issue : on a batch size of 1, the average speed of the model is around 180 it/s, but with a batch size of 2, the speed is about 60 it/s ! I cant find the origin of this behaviour (the allocated GPU memory stays about the same during the whole process). I am wondering if this kind of change is to be expected ? How can it be improved ?
By the way the script I use to measure the speed is the following :
num_kp = 18
   for batch_size in [1, 2]:
       mod = gcn.STGCN(gr, use_attention=False, use_tem=False, num_kpts=num_kp, num_people=num_p).eval().cuda()
       model_input = torch.rand((batch_size, 3, 150, num_kp, 1)).cuda()

       with torch.no_grad():
           for _ in tqdm.tqdm(range(100)):
               mod(model_input)


Thanks !