tazz:

the whole idea was to run inference in pytorch and compare it to the AOT bundle that is generated using glow.


Sure, but you don’t need to use an AOT bundle to do so. If you go through the torch_glow path, nothing is AOT, it’s done compiled just-in-time.



 tazz:

I did have a question if I had to compare execution of both on cuda gpu what would be a good way to get some results?


We don’t have support for CUDA right now directly. Theoretically you could try to target CUDA via our LLVM backend, but it’s been much more targeted toward CPUs, and I don’t know of anyone who has tried this. We also have an OpenCL backend, but it’s not a big focus of ours.
In open source we have focused more on inference accelerators backends like NNPI and Habana. There are also open-source users that contribute PRs usually targeted for LLVM-based backends, but those backends are all kept private by those contributors.