right now the observer uses fixed range depending on dtype: https://github.com/pytorch/pytorch/blob/master/torch/quantization/observer.py#L153-L162 6, feel free to add the support for quant_min and quant_max for observer.
cc @raghuramank100 for sub 8 bit observer support.