Tiru_B:

How can I use a torch.save and torch.load model on a quantized model?



How can I use a torch.save and torch.load model on a quantized model?

Currently we only support torch.save(model.state_dict()) and model.load_state_dict(…) I think. torch.save/torch.load model directly is not yet supported I believe.

Will the entire state dict have same scale and zero points?

No,  they’ll have scale/zero_point that’s calculated from the calibration step.

How can I get each layer scale and zero points from the quantized model?

you can print the quantized model and it will show scale and zero_point, e.g.:
> print(torch.nn.quantized.Conv2d(3, 3, 3))
QuantizedConv2d(3, 3, kernel_size=(3, 3), stride=(1, 1), scale=1.0, zero_point=0)