After fixing an issue with squeeze (changed it to .squeeze(1)) I get 742.706us for a single forward pass. Since the time is really low due to the small workload, you won’t see any speedup unfortunately.
If your workload is a bit higher (e.g. resnet50), you should see a speedup of approx. 2x.