Please see quant docs: add common errors section by vkuzo · Pull Request #49902 · pytorch/pytorch · GitHub 107 which was landed recently and adds documentation about this error.  It looks like one of your layers expects an int8 tensor, but it is being passed an fp32 tensor.  The fix would be to figure out where exactly in your model this is happening (it’s in the stack trace), and then add a QuantStub right before it.  Also, the code which prepares and converts the model needs to include the QuantStub objects, as those are a part of the quantization flow.