Hey all,
I’ve been experimenting with quantization aware training using pytorch 1.3.
I managed to adapt my model as demonstrated in the tutorial.
The documenation mentions that fake quantization is possible on GPU, however I notice that it is extremely slow.
Monitoring nvidia-smi shows that I only use 7% of the GPU, while it is close to 100% when using the non-qat adapted model.
Is this expected or am I doing something wrong?