Here is another sample, this is the model output when I removed all PReLUs and used ReLUs isntead(incase it was a hinderance):
Size (MB): 87.205847
QConfig(activation=functools.partial(<class 'torch.quantization.observer.HistogramObserver'>, reduce_range=True), weight=functools.partial(<class 'torch.quantization.observer.PerChannelMinMaxObserver'>, dtype=torch.qint8, qscheme=torch.per_channel_symmetric))
Model after quantization(converted-prepared): ResNet(
  (conv1): ConvReLU2d(
    (0): Conv2d(
      3, 64, kernel_size=(3, 3), stride=(1, 1)
      (activation_post_process): HistogramObserver()
    )
    (1): ReLU(
      (activation_post_process): HistogramObserver()
    )
  )
  (bn1): Identity()
  (prelu): PReLU(num_parameters=1)
  (prelu_q): PReLU_Quantized(
    (quantized_op): FloatFunctional(
      (activation_post_process): HistogramObserver()
    )
    (quant): QuantStub(
      (activation_post_process): HistogramObserver()
    )
    (dequant): DeQuantStub()
  )
  (reluooo): Identity()
  (maxpool): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  (layer1): Sequential(
    (0): IRBlock(
      (bn0): BatchNorm2d(
        64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True
        (activation_post_process): HistogramObserver()
      )
      (conv1): ConvReLU2d(
        (0): Conv2d(
          64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)
          (activation_post_process): HistogramObserver()
        )
        (1): ReLU(
          (activation_post_process): HistogramObserver()
        )
      )
      (bn1): Identity()
      (prelu): PReLU(num_parameters=1)
      (prelu_q): PReLU_Quantized(
        (quantized_op): FloatFunctional(
          (activation_post_process): HistogramObserver()
        )
        (quant): QuantStub(
          (activation_post_process): HistogramObserver()
        )
        (dequant): DeQuantStub()
      )
      (reluooo): Identity()
      (conv2): Conv2d(
        64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)
        (activation_post_process): HistogramObserver()
      )
      (bn2): Identity()
      (se): SEBlock(
        (avg_pool): AdaptiveAvgPool2d(output_size=1)
        (mult_xy): FloatFunctional(
          (activation_post_process): HistogramObserver()
        )
        (fc): Sequential(
          (0): Linear(
            in_features=64, out_features=4, bias=True
            (activation_post_process): HistogramObserver()
          )
          (1): PReLU(num_parameters=1)
          (2): Linear(
            in_features=4, out_features=64, bias=True
            (activation_post_process): HistogramObserver()
          )
          (3): Sigmoid()
        )
        (fc1): Linear(
          in_features=64, out_features=4, bias=True
          (activation_post_process): HistogramObserver()
        )
        (prelu): PReLU(num_parameters=1)
        (fc2): Linear(
          in_features=4, out_features=64, bias=True
          (activation_post_process): HistogramObserver()
        )
        (sigmoid): Sigmoid()
      )
      (add_residual_relu): FloatFunctional(
        (activation_post_process): HistogramObserver()
      )
    )
    (1): IRBlock(
      (bn0): BatchNorm2d(
        64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True
        (activation_post_process): HistogramObserver()
      )
      (conv1): ConvReLU2d(
        (0): Conv2d(
          64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)
          (activation_post_process): HistogramObserver()
        )
        (1): ReLU(
          (activation_post_process): HistogramObserver()
        )
      )
      (bn1): Identity()
      (prelu): PReLU(num_parameters=1)
      (prelu_q): PReLU_Quantized(
        (quantized_op): FloatFunctional(
          (activation_post_process): HistogramObserver()
        )
        (quant): QuantStub(
          (activation_post_process): HistogramObserver()
        )
        (dequant): DeQuantStub()
      )
      (reluooo): Identity()
      (conv2): Conv2d(
        64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)
        (activation_post_process): HistogramObserver()
      )
      (bn2): Identity()
      (se): SEBlock(
        (avg_pool): AdaptiveAvgPool2d(output_size=1)
        (mult_xy): FloatFunctional(
          (activation_post_process): HistogramObserver()
        )
        (fc): Sequential(
          (0): Linear(
            in_features=64, out_features=4, bias=True
            (activation_post_process): HistogramObserver()
          )
          (1): PReLU(num_parameters=1)
          (2): Linear(
            in_features=4, out_features=64, bias=True
            (activation_post_process): HistogramObserver()
          )
          (3): Sigmoid()
        )
        (fc1): Linear(
          in_features=64, out_features=4, bias=True
          (activation_post_process): HistogramObserver()
        )
        (prelu): PReLU(num_parameters=1)
        (fc2): Linear(
          in_features=4, out_features=64, bias=True
          (activation_post_process): HistogramObserver()
        )
        (sigmoid): Sigmoid()
      )
      (add_residual_relu): FloatFunctional(
        (activation_post_process): HistogramObserver()
      )
    )
  )
  (layer2): Sequential(
    (0): IRBlock(
      (bn0): BatchNorm2d(
        64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True
        (activation_post_process): HistogramObserver()
      )
      (conv1): ConvReLU2d(
        (0): Conv2d(
          64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)
          (activation_post_process): HistogramObserver()
        )
        (1): ReLU(
          (activation_post_process): HistogramObserver()
        )
      )
      (bn1): Identity()
      (prelu): PReLU(num_parameters=1)
      (prelu_q): PReLU_Quantized(
        (quantized_op): FloatFunctional(
          (activation_post_process): HistogramObserver()
        )
        (quant): QuantStub(
          (activation_post_process): HistogramObserver()
        )
        (dequant): DeQuantStub()
      )
      (reluooo): Identity()
      (conv2): Conv2d(
        64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1)
        (activation_post_process): HistogramObserver()
      )
      (bn2): Identity()
      (downsample): Sequential(
        (0): Conv2d(
          64, 128, kernel_size=(1, 1), stride=(2, 2)
          (activation_post_process): HistogramObserver()
        )
        (1): Identity()
      )
      (se): SEBlock(
        (avg_pool): AdaptiveAvgPool2d(output_size=1)
        (mult_xy): FloatFunctional(
          (activation_post_process): HistogramObserver()
        )
        (fc): Sequential(
          (0): Linear(
            in_features=128, out_features=8, bias=True
            (activation_post_process): HistogramObserver()
          )
          (1): PReLU(num_parameters=1)
          (2): Linear(
            in_features=8, out_features=128, bias=True
            (activation_post_process): HistogramObserver()
          )
          (3): Sigmoid()
        )
        (fc1): Linear(
          in_features=128, out_features=8, bias=True
          (activation_post_process): HistogramObserver()
        )
        (prelu): PReLU(num_parameters=1)
        (fc2): Linear(
          in_features=8, out_features=128, bias=True
          (activation_post_process): HistogramObserver()
        )
        (sigmoid): Sigmoid()
      )
      (add_residual_relu): FloatFunctional(
        (activation_post_process): HistogramObserver()
      )
    )
    (1): IRBlock(
      (bn0): BatchNorm2d(
        128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True
        (activation_post_process): HistogramObserver()
      )
      (conv1): ConvReLU2d(
        (0): Conv2d(
          128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)
          (activation_post_process): HistogramObserver()
        )
        (1): ReLU(
          (activation_post_process): HistogramObserver()
        )
      )
      (bn1): Identity()
      (prelu): PReLU(num_parameters=1)
      (prelu_q): PReLU_Quantized(
        (quantized_op): FloatFunctional(
          (activation_post_process): HistogramObserver()
        )
        (quant): QuantStub(
          (activation_post_process): HistogramObserver()
        )
        (dequant): DeQuantStub()
      )
      (reluooo): Identity()
      (conv2): Conv2d(
        128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)
        (activation_post_process): HistogramObserver()
      )
      (bn2): Identity()
      (se): SEBlock(
        (avg_pool): AdaptiveAvgPool2d(output_size=1)
        (mult_xy): FloatFunctional(
          (activation_post_process): HistogramObserver()
        )
        (fc): Sequential(
          (0): Linear(
            in_features=128, out_features=8, bias=True
            (activation_post_process): HistogramObserver()
          )
          (1): PReLU(num_parameters=1)
          (2): Linear(
            in_features=8, out_features=128, bias=True
            (activation_post_process): HistogramObserver()
          )
          (3): Sigmoid()
        )
        (fc1): Linear(
          in_features=128, out_features=8, bias=True
          (activation_post_process): HistogramObserver()
        )
        (prelu): PReLU(num_parameters=1)
        (fc2): Linear(
          in_features=8, out_features=128, bias=True
          (activation_post_process): HistogramObserver()
        )
        (sigmoid): Sigmoid()
      )
      (add_residual_relu): FloatFunctional(
        (activation_post_process): HistogramObserver()
      )
    )
  )
  (layer3): Sequential(
    (0)
Post Training Quantization Prepare: Inserting Observers

 Inverted Residual Block:After observer insertion

 ConvReLU2d(
  (0): Conv2d(
    3, 64, kernel_size=(3, 3), stride=(1, 1)
    (activation_post_process): HistogramObserver()
  )
  (1): ReLU(
    (activation_post_process): HistogramObserver()
  )
)

This is the error I get when using this model (above):
--------------------------
Traceback (most recent call last):
  File "d:\Codes\org\python\Quantization\quantizer.py", line 270, in <module>
    test_the_model(True)
  File "d:\Codes\org\python\Quantization\quantizer.py", line 218, in test_the_model
    check_and_tell(model, pic1, pic2)
  File "d:\Codes\org\python\Quantization\quantizer.py", line 203, in check_and_tell
    embd1 = model(img1.unsqueeze(0))
  File "C:\Users\User\Anaconda3\Lib\site-packages\torch\nn\modules\module.py", line 726, in _call_impl
    result = self.forward(*input, **kwargs)
  File "d:\codes\org\python\FV\quantized_models.py", line 599, in forward
    x = self.bn3(x)
  File "C:\Users\User\Anaconda3\Lib\site-packages\torch\nn\modules\module.py", line 726, in _call_impl
    result = self.forward(*input, **kwargs)
  File "C:\Users\User\Anaconda3\Lib\site-packages\torch\nn\modules\batchnorm.py", line 136, in forward
    self.weight, self.bias, bn_training, exponential_average_factor, self.eps)
  File "C:\Users\User\Anaconda3\Lib\site-packages\torch\nn\functional.py", line 2039, in batch_norm
    training, momentum, eps, torch.backends.cudnn.enabled
RuntimeError: Could not run 'aten::native_batch_norm' with arguments from the 'QuantizedCPU' backend. 'aten::native_batch_norm' is only available for these backends: [CPU, MkldnnCPU, BackendSelect, Named, Autograd, Profiler, Tracer, Autocast, Batched].

CPU: registered at aten\src\ATen\CPUType.cpp:1594 [kernel]
MkldnnCPU: registered at aten\src\ATen\MkldnnCPUType.cpp:139 [kernel]
BackendSelect: fallthrough registered at ..\aten\src\ATen\core\BackendSelectFallbackKernel.cpp:3 [backend fallback]
Named: registered at ..\aten\src\ATen\core\NamedRegistrations.cpp:7 [backend fallback]
Autograd: registered at ..\torch\csrc\autograd\generated\VariableType_0.cpp:7879 [kernel]
Profiler: registered at ..\torch\csrc\autograd\generated\ProfiledType_0.cpp:2050 [kernel]
Tracer: registered at ..\torch\csrc\autograd\generated\TraceType_0.cpp:8256 [kernel]
Autocast: fallthrough registered at ..\aten\src\ATen\autocast_mode.cpp:375 [backend fallback]
Batched: registered at ..\aten\src\ATen\BatchingRegistrations.cpp:149 [backend fallback]