Updated to the latest nighly(1.7.0.dev20200714+cpu and torchvision-0.8.0.dev20200714+cpu) just now , it got a bit further, but ultimately crashed with the same error :
Size (MB): 89.322487
QConfig(activation=functools.partial(<class 'torch.quantization.observer.HistogramObserver'>, reduce_range=True), weight=functools.partial(<class 'torch.quantization.observer.PerChannelMinMaxObserver'>, dtype=torch.qint8, qscheme=torch.per_channel_symmetric))
Post Training Quantization Prepare: Inserting Observers

 Inverted Residual Block:After observer insertion

 Conv2d(
  3, 64, kernel_size=(3, 3), stride=(1, 1)
  (activation_post_process): HistogramObserver()
)
<inside se forward:>
X: tensor([[-1.5691, -0.7516, -0.7360, -0.6458]])
--------------------------
<inside se forward:>
X: tensor([[ 3.6604e-01,  3.3855e+00, -5.0032e-19, -9.0280e-19]])
--------------------------
<inside se forward:>
X: tensor([[-1.0513, -0.0656, -0.4529,  0.0653, -0.4762, -0.6304, -1.5043, -0.9484]])
--------------------------
<inside se forward:>
X: tensor([[ 4.8730,  1.6650, -0.5135, -0.6811, -0.0392, -0.4689, -0.1496,  0.0717]])
--------------------------
<inside se forward:>
X: tensor([[-1.8759, -0.8886, -1.3295, -0.5375,  0.7598, -0.8526, -1.9066,  0.0985,
         -0.1461, -0.5857,  0.1513, -0.3050,  0.1955, -0.8470,  0.4528,  0.9689]])
--------------------------
<inside se forward:>
X: tensor([[ 1.6184e+00, -2.2714e-18,  2.8052e+00,  1.0378e+01,  4.6361e-05,
          1.0644e+01,  1.4302e-02,  2.6143e-02,  2.4926e-05,  6.2237e+00,
          8.8411e-05,  6.4360e+00,  3.3530e+00,  3.9302e-05,  8.1652e+00,
          8.7950e-07]])
--------------------------
<inside se forward:>
X: tensor([[ 9.1687e+00,  3.1469e+00, -1.1788e+01,  4.9410e-02,  1.7272e+00,
         -3.0913e+00,  1.1572e+00, -6.7104e+00,  1.1371e+01,  4.8926e+00,
         -1.3102e+00, -4.9773e+00, -4.1444e+00, -6.3367e-01, -1.5672e+00,
          4.2629e+00,  3.2491e+00, -4.6632e+00,  5.9241e-01, -2.4883e+00,
          5.2599e+00, -7.1710e+00,  4.7197e+00,  7.2724e+00, -2.3363e+00,
         -2.2564e+00,  5.4431e+00, -2.2832e-12,  1.9732e+00,  1.1682e+00,
          6.1555e+00,  6.3574e+00]])
--------------------------
<inside se forward:>
X: tensor([[ 1.2785e-01,  1.1057e+00,  3.1581e-07,  9.7595e-01,  9.7386e-03,
          8.4260e-07,  2.4243e-01,  2.1749e+00,  4.5704e-01,  2.9307e+00,
          3.2384e+00,  2.6099e+00,  1.7640e-01,  4.3206e-04,  9.9380e-18,
          1.3450e-11,  1.5721e-09,  2.7632e-07,  3.6721e-04,  2.1237e-07,
          1.8839e-10,  1.8423e-02,  1.8514e-13,  4.3584e+00,  1.0972e-01,
          7.5909e-03,  4.3828e-02,  2.9285e-02,  8.3840e-07, -2.6420e-19,
          3.6933e-01,  1.0561e+00]])
--------------------------
0-feature dims: torch.Size([1, 512])
<inside se forward:>
X: tensor([[-1.5517, -0.8007, -0.7286, -0.6478]])
--------------------------
<inside se forward:>
X: tensor([[ 5.0945e-01,  3.2514e+00, -5.2950e-19, -9.1256e-19]])
Traceback (most recent call last):
  File "d:\Codes\org\python\Quantization\quantizer.py", line 266, in <module>
    quantize_test()
  File "d:\Codes\org\python\Quantization\quantizer.py", line 248, in quantize_test
    evaluate(model, dtloader, neval_batches=num_calibration_batches)
  File "d:\Codes\org\python\Quantization\quantizer.py", line 152, in evaluate
    features = model(image.unsqueeze(0))
  File "C:\Users\User\Anaconda3\Lib\site-packages\torch\nn\modules\module.py", line 726, in _call_impl
    result = self.forward(*input, **kwargs)
  File "d:\codes\org\python\FV\quantized_models.py", line 576, in forward
    x = self.layer1(x)
  File "C:\Users\User\Anaconda3\Lib\site-packages\torch\nn\modules\module.py", line 726, in _call_impl
    result = self.forward(*input, **kwargs)
  File "C:\Users\User\Anaconda3\Lib\site-packages\torch\nn\modules\container.py", line 117, in forward
    input = module(input)
  File "C:\Users\User\Anaconda3\Lib\site-packages\torch\nn\modules\module.py", line 726, in _call_impl
    result = self.forward(*input, **kwargs)
  File "d:\codes\org\python\FV\quantized_models.py", line 489, in forward
    out = self.se(out)
  File "C:\Users\User\Anaconda3\Lib\site-packages\torch\nn\modules\module.py", line 726, in _call_impl
    result = self.forward(*input, **kwargs)
  File "d:\codes\org\python\FV\quantized_models.py", line 447, in forward
    y = self.prelu_q(y)
  File "C:\Users\User\Anaconda3\Lib\site-packages\torch\nn\modules\module.py", line 726, in _call_impl
    result = self.forward(*input, **kwargs)
  File "d:\codes\org\python\FV\quantized_models.py", line 322, in forward
    inputs = self.quantized_op.add(torch.relu(inputs), weight_min_res)
  File "C:\Users\User\Anaconda3\Lib\site-packages\torch\nn\quantized\modules\functional_modules.py", line 46, in add
    r = self.activation_post_process(r)
  File "C:\Users\User\Anaconda3\Lib\site-packages\torch\nn\modules\module.py", line 726, in _call_impl
    result = self.forward(*input, **kwargs)
  File "C:\Users\User\Anaconda3\Lib\site-packages\torch\quantization\observer.py", line 862, in forward
    self.bins)
  File "C:\Users\User\Anaconda3\Lib\site-packages\torch\quantization\observer.py", line 813, in _combine_histograms
    histogram_with_output_range = torch.zeros((Nbins * downsample_rate), device=orig_hist.device)
RuntimeError: Trying to create tensor with negative dimension -4398046511104: [-4398046511104]