@0Chen I am facing the same issue too! Does anyone know how to perform inference after quantizing the model?? When I try to get the output, I get this error
Error(s) in loading state_dict for Module:
	 ('Copying from quantized Tensor to non-quantized Tensor is not allowed, please use dequantize to get a float Tensor from a quantized Tensor',)