I am aware of the synchronization. I measure by averaging over 100 forward passes (no difference with/without synchronization). If I use with torch.jit.optimized_execution(False): I get similar time as the original model.
model = torchvision.models.resnet50(pretrained=False).cuda()
model.eval()
model = torch.jit.script(model)

# Disable all gradient computations 
torch.set_grad_enabled(False)

# Load & transform the image

# warm-up
embedding = model(image)

t1 = time.time()
N = 100
for i in range(N):
   embedding = model(image)
t2 = time.time()
t = t2 - t1
print('Time: {:.4f}, avg: {:.4f}'.format(t, t / N))

I wonder if TorchScript is sensitive to GPU architecture (faster on the recent GPUs, but slower on old GPUs?). I have not done the same comparison on the new GPUs yet (in any case, I am interested in the old GPUs as they will be used in prod).