Performance of the compiled model is worse comparing with the original pytorch model. Is it expected? Have I missed some optimization steps? Should I use another way to get better performance?


PyTorch works hard to implement optimized kernels for CPU. In Glow, we haven’t spent much time recently working on optimizing our Conv kernels for our CPU backend. We have focused much more on custom HW AI accelerators such as the NNPI and Habana backends in Glow. IIRC, last we focused on CPU perf a couple years ago it was originally tested on a few specific models of x86 CPUs. So your perf is going to depend on what CPU you’re using.


How can I run multi-core inference using Glow with CPU backend?


We don’t currently support this. See prior discussion here: https://github.com/pytorch/glow/issues/1749 16