For static quantization, in addition to using the qconfig you also need to add Quant/Dequant Stubs around the modules you want quantized.
The tutorial (beta) Static Quantization with Eager Mode in PyTorch — PyTorch Tutorials 1.8.1+cu102 documentation 6 has more details on how to do so.