this error message means that you are trying to give a fp32 tensor to a quantized layer.  The way to fix it is to use QuantStub and DeQuantStub to control the conversions, feel free to check out the examples on https://pytorch.org/docs/stable/quantization.html 21 .