@singularity thanks for sharing. Is the entire network quantized or are there some layers running in float? If you can reproduce the behavior on server (using qnnpack) then you can use autograd profiler to get an op level breakdown to see which ops are causing the most slowdown.
It might also be easier to debug accuracy issue on the server in case the quantization noise is reproducible there.