to use eager mode quantization you will need to explicitly place QuantStub/DeQuantStub in the points where quantization needs to happen, could you follow (beta) Static Quantization with Eager Mode in PyTorch — PyTorch Tutorials 1.9.0+cu102 documentation 9 to quantize your model?
Alternatively, you can also try out our fx graph mode quantization: (prototype) FX Graph Mode Post Training Static Quantization — PyTorch Tutorials 1.9.0+cu102 documentation 4