It appears that the issue is in pytorch’s method for computing per_channel quantization parameters – instead of parallelizing the computation, it runs in a loop per layer. Since this is sequential, it bottlenecks the performance. For PerChannelMinMaxObserver (and the moving average version) it should be pretty easy to modify the code to run in parallel.