Update
I had a typo in my code which was causing the error. I debugged the code from start and was able to quantize deeplabv3_resnet50() model. The peculiar thing is that the quantized model is at best of same inference speed compared to the float model.