@aprasad: If you are just looking to quantize weights only and want to keep activations in fp32, please look into dynamic quantization. It does exactly that and you it will calculate scale/zero_points automatically for you as well.