The example listed in Quantization — PyTorch 1.7.1 documentation 13 shows how to use quant/dequant stubs to statically quantize the model using eager mode.
The error you are seeing is probably because you are missing a quant stub operator before the conv operator.