I have compiled an SSD-based object detection model in PyTorch with torch.jit.script(model). I benchmarked the scripted and the original models on Tesla K80 GPU (AWS p2 instance). Looks like the scripted model is slower than the original model.
Averaged over 100 images:
Original model: 0.1787 seconds per image
Scripted model: 0.1928 seconds per image
I also benchmarked a ResNet50 model, got similar slow-down.
Original ResNet50: 0.0281
Scripted ResNet50: 0.0303
I was expecting some speed-up, and disappointed by the slow-down.
Is this normal, or could I have missed something?