With quantization support in pytorch 1.3, this should work if you follow the flow in the pytorch tutorials for quantization: https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html 193