You can use API torch.jit.save() to save quantized models. Just as what has been done in PyTorch Quantization Tutorial. (beta) Static Quantization with Eager Mode in PyTorch — PyTorch Tutorials 1.9.1+cu102 documentation 47