Hi, thanks for your suggestions.
Recompiling from source improved the speed a bit, however it’s still 2.8 times slower (note that tensorflow was not compiled from source, and does not support some CPU flags).
Regarding NNPACK, there is no documentation on how to compile with NNPACK support.
So far I’m trying just to provide NNPACK_ROOT_DIR environment variable, but it does not work. I saw your question, did you manage to compile it?
Regarding JIT compiler, I do not think it will help, since python code is not the bottleneck here.