I have relatively large network fully exported to torchscript.
I reimport the torchscript in python, and then I need to run the forward pass a number of times on a set of frames.
The weird thing is that while the first forward pass runs at an acceptable speed, the 2nd one (and only the second one, after the first pass has returned the results to CPU) seems to be tacking forever, more than 50 seconds.
I’ve even tried to call
torch.cuda.synchronize(device=self.device)

after the .to('cpu').
The rest of the frames are processed at reasonable speed, it’s really only the 2nd one   what could it be doing in this 2nd pass?