<p dir="auto">(This is a feature that I've been working on (see <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="453247002" data-permission-text="Title is private" data-url="https://github.com/pytorch/pytorch/issues/21493" data-hovercard-type="pull_request" data-hovercard-url="/pytorch/pytorch/pull/21493/hovercard?comment_id=500513129&amp;comment_type=issue_comment" href="https://github.com/pytorch/pytorch/pull/21493#issuecomment-500513129">#21493 (comment)</a>). I open this issue so that relevant PRs can be better tracked.)</p>
<h2 dir="auto"><g-emoji class="g-emoji" alias="rocket" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f680.png">🚀</g-emoji> Feature and Motivation</h2>
<p dir="auto">Better CMake integration. Currently there are a lot of inconsistency and redundancy between the Python setup scripts and CMake scripts (duplicated env check, overriding env check, inconsistent treatment of which results of checking will be used, etc.). I would like to improve the integration.</p>
<h2 dir="auto">Pitch</h2>
<p dir="auto">The improvement I would like to implement includes:</p>
<ul dir="auto">
<li>Allowing build options to be directly specified using options specified in CMakeLists.txt (without explicitly being processed in Python setup scripts).</li>
<li>Allowing users to adjust build options directly via cmake-gui/ccmake/editing CMakeCache.txt (currently users must take care of both options in CMakeCache.txt and environmental variables passed to setup.py and ensure they are consistent upon rebuild).</li>
<li>removing a lot of system and library environment checks in Python setup scripts while CMake can already handle them pretty well.</li>
<li>...</li>
</ul>
<h2 dir="auto">Existing PRs (including those that were made to clear the path)</h2>
<p dir="auto"><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="449423118" data-permission-text="Title is private" data-url="https://github.com/pytorch/pytorch/issues/21034" data-hovercard-type="pull_request" data-hovercard-url="/pytorch/pytorch/pull/21034/hovercard" href="https://github.com/pytorch/pytorch/pull/21034">#21034</a><br>
<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="451631812" data-permission-text="Title is private" data-url="https://github.com/pytorch/pytorch/issues/21296" data-hovercard-type="pull_request" data-hovercard-url="/pytorch/pytorch/pull/21296/hovercard" href="https://github.com/pytorch/pytorch/pull/21296">#21296</a><br>
<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="452638613" data-permission-text="Title is private" data-url="https://github.com/pytorch/pytorch/issues/21417" data-hovercard-type="pull_request" data-hovercard-url="/pytorch/pytorch/pull/21417/hovercard" href="https://github.com/pytorch/pytorch/pull/21417">#21417</a><br>
<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="453247002" data-permission-text="Title is private" data-url="https://github.com/pytorch/pytorch/issues/21493" data-hovercard-type="pull_request" data-hovercard-url="/pytorch/pytorch/pull/21493/hovercard" href="https://github.com/pytorch/pytorch/pull/21493">#21493</a><br>
<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="454910949" data-permission-text="Title is private" data-url="https://github.com/pytorch/pytorch/issues/21653" data-hovercard-type="pull_request" data-hovercard-url="/pytorch/pytorch/pull/21653/hovercard" href="https://github.com/pytorch/pytorch/pull/21653">#21653</a><br>
<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="457176905" data-permission-text="Title is private" data-url="https://github.com/pytorch/pytorch/issues/21877" data-hovercard-type="pull_request" data-hovercard-url="/pytorch/pytorch/pull/21877/hovercard" href="https://github.com/pytorch/pytorch/pull/21877">#21877</a><br>
<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="460049359" data-permission-text="Title is private" data-url="https://github.com/pytorch/pytorch/issues/22153" data-hovercard-type="pull_request" data-hovercard-url="/pytorch/pytorch/pull/22153/hovercard" href="https://github.com/pytorch/pytorch/pull/22153">#22153</a><br>
<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="460049393" data-permission-text="Title is private" data-url="https://github.com/pytorch/pytorch/issues/22154" data-hovercard-type="pull_request" data-hovercard-url="/pytorch/pytorch/pull/22154/hovercard" href="https://github.com/pytorch/pytorch/pull/22154">#22154</a><br>
<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="460568396" data-permission-text="Title is private" data-url="https://github.com/pytorch/pytorch/issues/22215" data-hovercard-type="pull_request" data-hovercard-url="/pytorch/pytorch/pull/22215/hovercard" href="https://github.com/pytorch/pytorch/pull/22215">#22215</a><br>
<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="462248945" data-permission-text="Title is private" data-url="https://github.com/pytorch/pytorch/issues/22360" data-hovercard-type="pull_request" data-hovercard-url="/pytorch/pytorch/pull/22360/hovercard" href="https://github.com/pytorch/pytorch/pull/22360">#22360</a></p>
<h2 dir="auto"><g-emoji class="g-emoji" alias="bug" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f41b.png">🐛</g-emoji> Bug</h2>
<p dir="auto">I am getting above error while doing inference after quantization of my model.<br>
a. This happens after 62nd epoch. If I quantize using any weights till 62nd epoch, every thing is fine.<br>
b. If I use partial (1/3) data to train my model, I see no issue.<br>
c. Only with "qnnpack". No issue with 'fbgemm'<br>
d. After executing operation in model that cause this error, the output becomes random (due to "qnnp_delete_operator(convolution);" in qnnpack/convolution.c)<br>
e. Location of error (in model) changes with weights.</p>
<h2 dir="auto">To Reproduce</h2>
<p dir="auto">Steps to reproduce the behavior:</p>
<ol dir="auto">
<li>Train float model</li>
<li>Load trained model</li>
<li>Quantize float model with "qnnpack' backend: (my quantization script)</li>
</ol>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="def static_quantize(model, backend):
    if backend not in torch.backends.quantized.supported_engines:
        raise RuntimeError(&quot;Quantized backend not supported &quot;)
    model.eval()
    if backend == 'fbgemm':
        torch.backends.quantized.engine = 'fbgemm'
        model.qconfig = torch.quantization.get_default_qconfig('fbgemm')
    elif backend == 'qnnpack':
        torch.backends.quantized.engine = 'qnnpack'
        model.qconfig = torch.quantization.get_default_qconfig('qnnpack')
 
    print(model.qconfig)
    model.fuse_model()
    torch.quantization.prepare(model, inplace=True)
    # Calibrate first
    print(&quot;Calibrating model...&quot;)
    epochs = 10
    validate(model, cfg, epochs)
    # calibration done
    torch.quantization.convert(model, inplace=True)
    # Test post quantization accuracy
    validate(model, cfg, epochs)                  &lt;------- **ERROR** seen  here
"><pre class="notranslate"><code class="notranslate">def static_quantize(model, backend):
    if backend not in torch.backends.quantized.supported_engines:
        raise RuntimeError("Quantized backend not supported ")
    model.eval()
    if backend == 'fbgemm':
        torch.backends.quantized.engine = 'fbgemm'
        model.qconfig = torch.quantization.get_default_qconfig('fbgemm')
    elif backend == 'qnnpack':
        torch.backends.quantized.engine = 'qnnpack'
        model.qconfig = torch.quantization.get_default_qconfig('qnnpack')
 
    print(model.qconfig)
    model.fuse_model()
    torch.quantization.prepare(model, inplace=True)
    # Calibrate first
    print("Calibrating model...")
    epochs = 10
    validate(model, cfg, epochs)
    # calibration done
    torch.quantization.convert(model, inplace=True)
    # Test post quantization accuracy
    validate(model, cfg, epochs)                  &lt;------- **ERROR** seen  here

</code></pre></div>
<ol start="4" dir="auto">
<li>Quantized model operation causing this issue</li>
</ol>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="    (3): Sequential(
      (0): QuantBlock(
        (skip_add): QFunctional(scale=0.11691034585237503, zero_point=0)
        (conv): Sequential(
          (0): ConvBN(
            (0): QuantizedConv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), scale=0.2153073400259018, zero_point=62, padding=(1, 1), groups=256)
            (1): Identity()
          )
          (1): ConvBNReLU(
            (0): QuantizedConvReLU2d(256, 256, kernel_size=(1, 1), stride=(1, 1), scale=0.06376750022172928, zero_point=0)
            (1): Identity()
            (2): Identity()
          )
        )
      )"><pre class="notranslate"><code class="notranslate">    (3): Sequential(
      (0): QuantBlock(
        (skip_add): QFunctional(scale=0.11691034585237503, zero_point=0)
        (conv): Sequential(
          (0): ConvBN(
            (0): QuantizedConv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), scale=0.2153073400259018, zero_point=62, padding=(1, 1), groups=256)
            (1): Identity()
          )
          (1): ConvBNReLU(
            (0): QuantizedConvReLU2d(256, 256, kernel_size=(1, 1), stride=(1, 1), scale=0.06376750022172928, zero_point=0)
            (1): Identity()
            (2): Identity()
          )
        )
      )
</code></pre></div>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="## Expected behavior
output of  qnnpack and fbgemm should be same. 


## Environment

 - PyTorch Version (e.g., 1.0): '1.5.0.dev20200316' (nightly)
 - OS (e.g., Linux): Ubuntu 18.04
 - How you installed PyTorch (`conda`, `pip`, source): Conda
 - Build command you used (if compiling from source): NA
 - Python version: 3.7
 - CUDA/cuDNN version: 10.1
 - GPU models and configuration: RTX 2060
 - Any other relevant information:

## Additional context

Working weights for Conv layer.
tensor([[[[-9.8111e-02, -2.3818e-01, -1.0097e-01],
          [-8.6748e-02, -1.5611e+00, -3.7088e-02],
          [-7.2189e-02,  9.3585e-03, -2.1457e-01]]],
        [[[-9.0585e-01, -1.4491e+00, -1.0290e+00],
          [ 6.0444e-02,  1.7321e-02,  4.5470e-02],
          [ 2.0289e-01,  1.7305e-01,  2.3026e-01]]],
        [[[-3.9760e-01, -1.7248e-01, -4.0785e-01],
          [ 5.5625e-03,  7.6404e-01, -7.1765e-02],
          [-5.1368e-01, -3.6436e-01, -5.2025e-01]]],
        [[[-1.2394e-01, -4.8440e-01, -4.7897e-01],
          [-4.6179e-01, -4.1148e-01, -4.7571e-01],
          [-3.6709e-01, -2.5656e-01, -3.8935e-01]]],
        [[[ 3.8117e-01,  2.7804e-01,  4.4873e-01],
          [ 4.8555e-01, -2.8842e-01,  2.9044e-01],
          [ 2.5739e-01,  6.3275e-01,  3.8192e-01]]],
        [[[-2.5214e-01,  3.7439e-01, -3.5673e-01],
          [ 3.0839e-01,  1.0794e+00,  4.3154e-01],
          [-2.5715e-01,  3.6404e-01, -3.6204e-01]]],
        [[[-5.4990e-01, -2.2252e-01, -5.0191e-01],
          [-1.7313e-01,  3.8547e-01, -1.1505e-01],
          [-6.0288e-01, -2.6323e-01, -6.4934e-01]]],
        [[[ 2.3457e-01,  2.6993e-01,  3.5980e-01],
          [ 1.4922e-01,  1.1058e+00,  4.5924e-01],
          [ 5.3947e-01,  1.6402e-01,  1.4880e-01]]],
        [[[ 5.9202e-01,  5.5206e-02,  5.2480e-01],
          [ 4.3953e-02, -1.1449e-01,  1.7012e-01],
          [ 5.4099e-01, -1.3535e-01,  4.9393e-01]]],
        [[[-3.6607e-01,  2.1279e-01, -3.4666e-01],
          [ 1.9444e-01,  1.1031e+00,  2.0701e-01],
          [-3.1785e-01,  2.0595e-01, -3.7019e-01]]],
        [[[ 9.1770e-02, -2.8905e-02,  1.8759e-01],
          [-1.2641e-01, -1.3670e+00,  1.0820e-01],
          [ 3.7640e-01,  2.5282e-02,  1.6139e-01]]],
        [[[ 6.6392e-02,  4.7435e-01,  2.2360e-01],
          [ 1.1640e-01,  4.4550e-02,  1.0114e-01],
          [ 7.5970e-01, -2.6593e-02,  7.4978e-01]]],
        [[[-2.4952e-01, -2.9136e-01, -4.3275e-01],
          [-4.3836e-01,  5.7868e-01, -4.1796e-01],
          [-3.1188e-01, -4.1504e-01, -6.0209e-01]]],
        [[[-7.6740e-03, -1.2746e-01,  7.3953e-02],
          [ 1.1245e-01,  2.4729e+00,  2.5669e-01],
          [ 2.8247e-02,  1.6268e-01, -7.0686e-04]]],
        [[[ 6.0586e-01, -1.2779e-02,  6.1001e-01],
          [-9.3175e-02, -3.9671e-01, -1.0740e-01],
          [ 6.3908e-01, -1.3216e-01,  5.9539e-01]]],
        [[[ 3.3967e-01, -4.4094e-01,  3.3106e-01],
          [-2.6672e-01,  1.2313e+00, -2.3391e-01],
          [ 2.3279e-01, -2.8770e-01,  3.1637e-01]]],
        [[[-3.4574e-01, -1.9476e-01, -4.8610e-01],
          [-2.2992e-01, -1.4230e-02, -1.9174e-01],
          [-5.4567e-01, -3.5120e-01, -6.8289e-01]]],
        [[[ 4.5144e-01, -2.5136e-02,  5.3568e-01],
          [ 3.1801e-02, -5.8331e-01, -3.4960e-02],
          [-9.1540e-04, -4.8772e-01, -1.8636e-01]]],
        [[[ 1.0150e-01, -1.2134e-02,  1.6102e-02],
          [-4.1484e-02,  4.3275e-01, -6.8248e-02],
          [-7.1185e-01, -7.1240e-01, -7.0851e-01]]],
        [[[-3.6291e-01,  1.8500e-01, -4.4183e-01],
          [ 8.4609e-02,  5.8029e-01,  1.4827e-01],
          [-5.9297e-01,  1.8420e-02, -6.8263e-01]]],
        [[[-5.8049e-01, -3.5364e-01, -7.2523e-01],
          [-3.7166e-01, -3.1462e-01, -5.6026e-01],
          [-2.5259e-01, -2.0127e-01, -4.1163e-01]]],
        [[[-4.1470e-01, -2.7154e-01, -3.6831e-01],
          [-1.1561e-01, -3.8793e-01, -2.6043e-01],
          [-6.1500e-01, -1.8974e-01, -6.4628e-01]]],
        [[[-4.7443e-01, -3.7602e-01, -4.7216e-01],
          [-3.7285e-01,  2.5685e-01, -1.8080e-01],
          [-3.2134e-01, -4.7332e-01, -4.5040e-01]]],
        [[[-2.4336e-01, -2.8814e-01, -2.2455e-01],
          [-2.7187e-01,  9.4440e-01, -3.6674e-01],
          [-2.5768e-01, -2.6512e-01, -2.8699e-01]]],
        [[[ 5.1769e-01, -3.3097e-02,  5.4375e-01],
          [-5.0911e-02,  5.1187e-01, -2.4701e-01],
          [ 5.3332e-01, -1.9179e-01,  6.5832e-01]]],
        [[[ 8.7317e-02, -2.7439e-01, -2.3370e-01],
          [-1.5015e-02, -1.7981e+00, -2.3003e-01],
          [-1.8798e-01, -2.9342e-01, -1.4052e-01]]],
        [[[-4.0463e-01, -5.8371e-01, -6.1616e-01],
          [-6.3553e-01,  2.8725e-01, -4.3908e-01],
          [ 4.6381e-03, -4.6432e-01,  7.3069e-02]]],
        [[[-6.9479e-01, -2.8610e-01, -6.3922e-01],
          [-3.1537e-02,  1.7567e-01, -4.9486e-04],
          [-3.4923e-01, -5.3215e-01, -3.3615e-01]]],
        [[[-6.9439e-01, -5.9017e-01, -7.6260e-01],
          [-1.4265e-01, -8.5820e-01, -1.9734e-01],
          [ 2.0202e-01,  2.4088e-01,  3.1906e-01]]],
        [[[ 4.8009e-01, -2.1032e-01,  4.4050e-01],
          [ 1.6932e-01,  3.0352e-01, -5.3674e-02],
          [ 5.3025e-01,  1.7536e-01,  6.8803e-01]]],
        [[[ 2.6775e-01,  8.2845e-01,  1.2976e-01],
          [ 4.5156e-01,  2.0780e-01,  2.3832e-01],
          [ 4.6654e-01,  2.3515e-01,  3.9078e-01]]],
        [[[ 4.9575e-01, -4.8354e-02,  6.2080e-01],
          [ 1.2933e-01,  4.5557e-01, -1.2404e-01],
          [ 5.6906e-01,  4.3322e-01,  4.7213e-01]]]])

Not working weights of Conv layer
tensor([[[[ 3.3279e-01, -2.0911e-01,  4.0976e-01],
          [-1.9482e-01, -2.8142e+00, -2.2299e-01],
          [ 3.9004e-01, -1.7411e-01,  4.2168e-01]]],
        [[[ 2.7489e-01,  9.7963e-01,  2.2939e-02],
          [ 7.0204e-01,  1.3131e+00,  5.6977e-01],
          [ 2.9443e-01,  8.0951e-01,  4.5278e-01]]],
        [[[ 8.4607e-01,  1.9752e-02,  1.0065e+00],
          [ 3.2082e-02, -2.1234e-01,  2.6094e-01],
          [ 1.0338e+00,  1.6526e-01,  9.6839e-01]]],
        [[[-2.5110e-01, -2.7901e-02, -2.9815e-01],
          [ 3.5116e-02,  2.2587e+00,  1.0436e-01],
          [-3.0516e-01, -1.5374e-02, -2.8045e-01]]],
        [[[-5.4234e-01,  7.3581e-02, -3.2796e-01],
          [ 4.3465e-02,  2.4696e+00, -1.4038e-02],
          [-2.6090e-01, -6.6863e-02, -2.2878e-01]]],
        [[[ 7.7577e-01,  1.6263e-01,  8.0620e-01],
          [ 1.2166e-01, -1.3349e+00,  2.2095e-02],
          [ 7.8446e-01,  1.8965e-01,  7.7734e-01]]],
        [[[ 2.8460e-01,  2.8625e-01,  2.8036e-01],
          [ 2.8995e-01, -2.0631e-01,  1.6038e-01],
          [ 2.2620e-01,  2.3574e-01,  4.5563e-01]]],
        [[[-8.2636e-01,  3.0475e-01, -4.1388e-01],
          [ 3.1593e-01,  1.9472e+00,  1.1006e-01],
          [-6.6613e-01,  2.9975e-01, -4.6172e-01]]],
        [[[-5.3745e-02, -1.6191e-01, -5.6559e-02],
          [-6.6030e-02, -5.9331e+00, -1.3726e-01],
          [-5.5640e-02, -1.1951e-01, -9.8351e-02]]],
        [[[-4.5139e-01,  3.0925e-01, -4.2166e-01],
          [ 1.5557e-01,  2.0291e+00,  9.6838e-02],
          [-5.0419e-01,  1.4905e-01, -4.8405e-01]]],
        [[[-5.9708e-02, -5.9127e-01,  2.7119e-01],
          [-7.0508e-01, -1.6482e+00, -6.5309e-01],
          [ 7.8688e-02, -1.0563e+00,  9.6375e-02]]],
        [[[ 7.2531e-01,  2.1859e-01,  8.1715e-01],
          [ 1.4592e-01, -1.3310e+00,  2.8370e-02],
          [ 7.2177e-01,  3.3575e-01,  7.5479e-01]]],
        [[[-6.9039e-01, -1.6698e-01, -8.5761e-01],
          [-7.3998e-02, -2.5820e-01, -3.2171e-02],
          [-9.9621e-01, -2.2918e-01, -1.1667e+00]]],
        [[[ 1.4400e-01,  1.5582e-01,  4.7007e-01],
          [ 6.2186e-03,  1.5049e+00,  1.3891e-01],
          [ 5.8765e-01, -9.2874e-02,  1.2079e+00]]],
        [[[-4.7601e-01,  1.7371e-01, -3.9373e-01],
          [-1.4278e-03,  2.3851e+00,  3.5437e-02],
          [-3.5845e-01,  7.5070e-02, -4.3677e-01]]
        [[[-3.9381e-01, -2.0094e-01, -3.3219e-01],
          [ 1.1169e-01,  1.8181e+00, -3.7194e-01],
          [-5.1897e-02,  1.7572e-01, -5.5085e-01]]],
        [[[ 8.1767e-01,  5.8831e-01,  7.3225e-01],
          [ 4.0319e-01, -8.8078e-01,  5.0771e-01],
          [ 8.4507e-01,  7.1296e-01,  7.6765e-01]]],
        [[[-6.0087e-02, -1.2139e+00, -1.1047e-03],
          [-6.6237e-01, -4.8200e-01, -8.6077e-01],
          [-1.8411e-01, -1.1920e+00, -7.4986e-02]]],
        [[[-9.4947e-01, -9.8611e-01, -1.8920e-01],
          [-8.7728e-02,  9.9390e-01, -3.6914e-01],
          [-7.8076e-01, -1.1616e-01, -6.4246e-01]]],
        [[[ 5.5130e-01, -3.4770e-01,  6.2945e-01],
          [-3.8659e-01, -1.4704e+00, -4.9395e-01],
          [ 5.3765e-01, -3.4119e-01,  6.7366e-01]]],
        [[[-4.4788e-01, -6.4880e-02,  5.1750e-01],
          [-6.9337e-01, -2.2426e+00, -1.9765e-02],
          [-6.0051e-01, -2.1296e-01,  2.9482e-01]]],
        [[[-2.9910e-01, -5.7485e-01, -5.7792e-01],
          [-6.2466e-01, -7.3714e-01, -7.8880e-01],
          [-7.6090e-01, -7.5296e-01, -1.1582e+00]]],
        [[[-9.3275e-01, -3.5397e-01, -9.9876e-01],
          [-2.0837e-01, -2.4731e-01, -2.6931e-01],
          [-9.3229e-01, -3.3563e-01, -8.7554e-01]]],
        [[[-7.8048e-01, -1.7040e-01, -8.7957e-01],
          [-1.0868e-01,  1.3689e+00, -1.6108e-01],
          [-8.2380e-01, -2.1541e-01, -8.2356e-01]]],
        [[[ 6.6214e-01, -2.8046e-01,  7.4298e-01],
          [-2.4776e-01,  2.3627e+00, -1.6467e-01],
          [ 5.6969e-01, -2.3224e-01,  6.5499e-01]]],
        [[[ 1.0392e+00,  7.8446e-02,  1.1745e+00],
          [-1.9321e-01, -5.8134e-01, -2.2323e-01],
          [ 7.3575e-01,  6.3480e-02,  1.2036e+00]]],
        [[[-7.4549e-01,  3.5171e-01, -7.4779e-01],
          [ 2.7267e-01, -1.0939e+00,  3.1744e-01],
          [-8.6094e-01,  1.8053e-01, -9.5986e-01]]],
        [[[-7.4761e-02, -7.4965e-02, -4.8395e-03],
          [-7.0916e-02, -9.4117e-02,  6.8479e-03],
          [-7.0671e-02, -5.7122e-02,  1.1384e-03]]],
        [[[ 2.0594e-02, -2.6332e-01, -2.0066e-01],
          [ 3.5086e-01,  2.8736e+00,  2.2054e-01],
          [ 1.2846e-01,  1.2652e-01, -2.1748e-01]]],
        [[[ 4.8172e-01,  4.0160e-01,  5.4478e-01],
          [ 4.1767e-01, -1.5756e+00,  4.4751e-01],
          [ 4.5373e-01,  4.2261e-01,  4.6070e-01]]],
        [[[-2.4105e-01,  6.2362e-02,  2.6139e-01],
          [-4.1651e-01, -1.4579e-01, -6.3829e-01],
          [-5.3446e-01, -7.7817e-01, -7.7269e-01]]],
        [[[ 9.1111e-02,  7.7725e-01,  7.9571e-01],
          [ 4.4018e-01,  1.0821e+00,  2.7830e-01],
          [ 5.1079e-01,  5.2820e-01,  4.4008e-01]]]])



cc @jerryzh168 @jianyuh @dzhulgakov @raghuramank100 @jamesr66a"><pre class="notranslate"><code class="notranslate">## Expected behavior
output of  qnnpack and fbgemm should be same. 


## Environment

 - PyTorch Version (e.g., 1.0): '1.5.0.dev20200316' (nightly)
 - OS (e.g., Linux): Ubuntu 18.04
 - How you installed PyTorch (`conda`, `pip`, source): Conda
 - Build command you used (if compiling from source): NA
 - Python version: 3.7
 - CUDA/cuDNN version: 10.1
 - GPU models and configuration: RTX 2060
 - Any other relevant information:

## Additional context

Working weights for Conv layer.
tensor([[[[-9.8111e-02, -2.3818e-01, -1.0097e-01],
          [-8.6748e-02, -1.5611e+00, -3.7088e-02],
          [-7.2189e-02,  9.3585e-03, -2.1457e-01]]],
        [[[-9.0585e-01, -1.4491e+00, -1.0290e+00],
          [ 6.0444e-02,  1.7321e-02,  4.5470e-02],
          [ 2.0289e-01,  1.7305e-01,  2.3026e-01]]],
        [[[-3.9760e-01, -1.7248e-01, -4.0785e-01],
          [ 5.5625e-03,  7.6404e-01, -7.1765e-02],
          [-5.1368e-01, -3.6436e-01, -5.2025e-01]]],
        [[[-1.2394e-01, -4.8440e-01, -4.7897e-01],
          [-4.6179e-01, -4.1148e-01, -4.7571e-01],
          [-3.6709e-01, -2.5656e-01, -3.8935e-01]]],
        [[[ 3.8117e-01,  2.7804e-01,  4.4873e-01],
          [ 4.8555e-01, -2.8842e-01,  2.9044e-01],
          [ 2.5739e-01,  6.3275e-01,  3.8192e-01]]],
        [[[-2.5214e-01,  3.7439e-01, -3.5673e-01],
          [ 3.0839e-01,  1.0794e+00,  4.3154e-01],
          [-2.5715e-01,  3.6404e-01, -3.6204e-01]]],
        [[[-5.4990e-01, -2.2252e-01, -5.0191e-01],
          [-1.7313e-01,  3.8547e-01, -1.1505e-01],
          [-6.0288e-01, -2.6323e-01, -6.4934e-01]]],
        [[[ 2.3457e-01,  2.6993e-01,  3.5980e-01],
          [ 1.4922e-01,  1.1058e+00,  4.5924e-01],
          [ 5.3947e-01,  1.6402e-01,  1.4880e-01]]],
        [[[ 5.9202e-01,  5.5206e-02,  5.2480e-01],
          [ 4.3953e-02, -1.1449e-01,  1.7012e-01],
          [ 5.4099e-01, -1.3535e-01,  4.9393e-01]]],
        [[[-3.6607e-01,  2.1279e-01, -3.4666e-01],
          [ 1.9444e-01,  1.1031e+00,  2.0701e-01],
          [-3.1785e-01,  2.0595e-01, -3.7019e-01]]],
        [[[ 9.1770e-02, -2.8905e-02,  1.8759e-01],
          [-1.2641e-01, -1.3670e+00,  1.0820e-01],
          [ 3.7640e-01,  2.5282e-02,  1.6139e-01]]],
        [[[ 6.6392e-02,  4.7435e-01,  2.2360e-01],
          [ 1.1640e-01,  4.4550e-02,  1.0114e-01],
          [ 7.5970e-01, -2.6593e-02,  7.4978e-01]]],
        [[[-2.4952e-01, -2.9136e-01, -4.3275e-01],
          [-4.3836e-01,  5.7868e-01, -4.1796e-01],
          [-3.1188e-01, -4.1504e-01, -6.0209e-01]]],
        [[[-7.6740e-03, -1.2746e-01,  7.3953e-02],
          [ 1.1245e-01,  2.4729e+00,  2.5669e-01],
          [ 2.8247e-02,  1.6268e-01, -7.0686e-04]]],
        [[[ 6.0586e-01, -1.2779e-02,  6.1001e-01],
          [-9.3175e-02, -3.9671e-01, -1.0740e-01],
          [ 6.3908e-01, -1.3216e-01,  5.9539e-01]]],
        [[[ 3.3967e-01, -4.4094e-01,  3.3106e-01],
          [-2.6672e-01,  1.2313e+00, -2.3391e-01],
          [ 2.3279e-01, -2.8770e-01,  3.1637e-01]]],
        [[[-3.4574e-01, -1.9476e-01, -4.8610e-01],
          [-2.2992e-01, -1.4230e-02, -1.9174e-01],
          [-5.4567e-01, -3.5120e-01, -6.8289e-01]]],
        [[[ 4.5144e-01, -2.5136e-02,  5.3568e-01],
          [ 3.1801e-02, -5.8331e-01, -3.4960e-02],
          [-9.1540e-04, -4.8772e-01, -1.8636e-01]]],
        [[[ 1.0150e-01, -1.2134e-02,  1.6102e-02],
          [-4.1484e-02,  4.3275e-01, -6.8248e-02],
          [-7.1185e-01, -7.1240e-01, -7.0851e-01]]],
        [[[-3.6291e-01,  1.8500e-01, -4.4183e-01],
          [ 8.4609e-02,  5.8029e-01,  1.4827e-01],
          [-5.9297e-01,  1.8420e-02, -6.8263e-01]]],
        [[[-5.8049e-01, -3.5364e-01, -7.2523e-01],
          [-3.7166e-01, -3.1462e-01, -5.6026e-01],
          [-2.5259e-01, -2.0127e-01, -4.1163e-01]]],
        [[[-4.1470e-01, -2.7154e-01, -3.6831e-01],
          [-1.1561e-01, -3.8793e-01, -2.6043e-01],
          [-6.1500e-01, -1.8974e-01, -6.4628e-01]]],
        [[[-4.7443e-01, -3.7602e-01, -4.7216e-01],
          [-3.7285e-01,  2.5685e-01, -1.8080e-01],
          [-3.2134e-01, -4.7332e-01, -4.5040e-01]]],
        [[[-2.4336e-01, -2.8814e-01, -2.2455e-01],
          [-2.7187e-01,  9.4440e-01, -3.6674e-01],
          [-2.5768e-01, -2.6512e-01, -2.8699e-01]]],
        [[[ 5.1769e-01, -3.3097e-02,  5.4375e-01],
          [-5.0911e-02,  5.1187e-01, -2.4701e-01],
          [ 5.3332e-01, -1.9179e-01,  6.5832e-01]]],
        [[[ 8.7317e-02, -2.7439e-01, -2.3370e-01],
          [-1.5015e-02, -1.7981e+00, -2.3003e-01],
          [-1.8798e-01, -2.9342e-01, -1.4052e-01]]],
        [[[-4.0463e-01, -5.8371e-01, -6.1616e-01],
          [-6.3553e-01,  2.8725e-01, -4.3908e-01],
          [ 4.6381e-03, -4.6432e-01,  7.3069e-02]]],
        [[[-6.9479e-01, -2.8610e-01, -6.3922e-01],
          [-3.1537e-02,  1.7567e-01, -4.9486e-04],
          [-3.4923e-01, -5.3215e-01, -3.3615e-01]]],
        [[[-6.9439e-01, -5.9017e-01, -7.6260e-01],
          [-1.4265e-01, -8.5820e-01, -1.9734e-01],
          [ 2.0202e-01,  2.4088e-01,  3.1906e-01]]],
        [[[ 4.8009e-01, -2.1032e-01,  4.4050e-01],
          [ 1.6932e-01,  3.0352e-01, -5.3674e-02],
          [ 5.3025e-01,  1.7536e-01,  6.8803e-01]]],
        [[[ 2.6775e-01,  8.2845e-01,  1.2976e-01],
          [ 4.5156e-01,  2.0780e-01,  2.3832e-01],
          [ 4.6654e-01,  2.3515e-01,  3.9078e-01]]],
        [[[ 4.9575e-01, -4.8354e-02,  6.2080e-01],
          [ 1.2933e-01,  4.5557e-01, -1.2404e-01],
          [ 5.6906e-01,  4.3322e-01,  4.7213e-01]]]])

Not working weights of Conv layer
tensor([[[[ 3.3279e-01, -2.0911e-01,  4.0976e-01],
          [-1.9482e-01, -2.8142e+00, -2.2299e-01],
          [ 3.9004e-01, -1.7411e-01,  4.2168e-01]]],
        [[[ 2.7489e-01,  9.7963e-01,  2.2939e-02],
          [ 7.0204e-01,  1.3131e+00,  5.6977e-01],
          [ 2.9443e-01,  8.0951e-01,  4.5278e-01]]],
        [[[ 8.4607e-01,  1.9752e-02,  1.0065e+00],
          [ 3.2082e-02, -2.1234e-01,  2.6094e-01],
          [ 1.0338e+00,  1.6526e-01,  9.6839e-01]]],
        [[[-2.5110e-01, -2.7901e-02, -2.9815e-01],
          [ 3.5116e-02,  2.2587e+00,  1.0436e-01],
          [-3.0516e-01, -1.5374e-02, -2.8045e-01]]],
        [[[-5.4234e-01,  7.3581e-02, -3.2796e-01],
          [ 4.3465e-02,  2.4696e+00, -1.4038e-02],
          [-2.6090e-01, -6.6863e-02, -2.2878e-01]]],
        [[[ 7.7577e-01,  1.6263e-01,  8.0620e-01],
          [ 1.2166e-01, -1.3349e+00,  2.2095e-02],
          [ 7.8446e-01,  1.8965e-01,  7.7734e-01]]],
        [[[ 2.8460e-01,  2.8625e-01,  2.8036e-01],
          [ 2.8995e-01, -2.0631e-01,  1.6038e-01],
          [ 2.2620e-01,  2.3574e-01,  4.5563e-01]]],
        [[[-8.2636e-01,  3.0475e-01, -4.1388e-01],
          [ 3.1593e-01,  1.9472e+00,  1.1006e-01],
          [-6.6613e-01,  2.9975e-01, -4.6172e-01]]],
        [[[-5.3745e-02, -1.6191e-01, -5.6559e-02],
          [-6.6030e-02, -5.9331e+00, -1.3726e-01],
          [-5.5640e-02, -1.1951e-01, -9.8351e-02]]],
        [[[-4.5139e-01,  3.0925e-01, -4.2166e-01],
          [ 1.5557e-01,  2.0291e+00,  9.6838e-02],
          [-5.0419e-01,  1.4905e-01, -4.8405e-01]]],
        [[[-5.9708e-02, -5.9127e-01,  2.7119e-01],
          [-7.0508e-01, -1.6482e+00, -6.5309e-01],
          [ 7.8688e-02, -1.0563e+00,  9.6375e-02]]],
        [[[ 7.2531e-01,  2.1859e-01,  8.1715e-01],
          [ 1.4592e-01, -1.3310e+00,  2.8370e-02],
          [ 7.2177e-01,  3.3575e-01,  7.5479e-01]]],
        [[[-6.9039e-01, -1.6698e-01, -8.5761e-01],
          [-7.3998e-02, -2.5820e-01, -3.2171e-02],
          [-9.9621e-01, -2.2918e-01, -1.1667e+00]]],
        [[[ 1.4400e-01,  1.5582e-01,  4.7007e-01],
          [ 6.2186e-03,  1.5049e+00,  1.3891e-01],
          [ 5.8765e-01, -9.2874e-02,  1.2079e+00]]],
        [[[-4.7601e-01,  1.7371e-01, -3.9373e-01],
          [-1.4278e-03,  2.3851e+00,  3.5437e-02],
          [-3.5845e-01,  7.5070e-02, -4.3677e-01]]
        [[[-3.9381e-01, -2.0094e-01, -3.3219e-01],
          [ 1.1169e-01,  1.8181e+00, -3.7194e-01],
          [-5.1897e-02,  1.7572e-01, -5.5085e-01]]],
        [[[ 8.1767e-01,  5.8831e-01,  7.3225e-01],
          [ 4.0319e-01, -8.8078e-01,  5.0771e-01],
          [ 8.4507e-01,  7.1296e-01,  7.6765e-01]]],
        [[[-6.0087e-02, -1.2139e+00, -1.1047e-03],
          [-6.6237e-01, -4.8200e-01, -8.6077e-01],
          [-1.8411e-01, -1.1920e+00, -7.4986e-02]]],
        [[[-9.4947e-01, -9.8611e-01, -1.8920e-01],
          [-8.7728e-02,  9.9390e-01, -3.6914e-01],
          [-7.8076e-01, -1.1616e-01, -6.4246e-01]]],
        [[[ 5.5130e-01, -3.4770e-01,  6.2945e-01],
          [-3.8659e-01, -1.4704e+00, -4.9395e-01],
          [ 5.3765e-01, -3.4119e-01,  6.7366e-01]]],
        [[[-4.4788e-01, -6.4880e-02,  5.1750e-01],
          [-6.9337e-01, -2.2426e+00, -1.9765e-02],
          [-6.0051e-01, -2.1296e-01,  2.9482e-01]]],
        [[[-2.9910e-01, -5.7485e-01, -5.7792e-01],
          [-6.2466e-01, -7.3714e-01, -7.8880e-01],
          [-7.6090e-01, -7.5296e-01, -1.1582e+00]]],
        [[[-9.3275e-01, -3.5397e-01, -9.9876e-01],
          [-2.0837e-01, -2.4731e-01, -2.6931e-01],
          [-9.3229e-01, -3.3563e-01, -8.7554e-01]]],
        [[[-7.8048e-01, -1.7040e-01, -8.7957e-01],
          [-1.0868e-01,  1.3689e+00, -1.6108e-01],
          [-8.2380e-01, -2.1541e-01, -8.2356e-01]]],
        [[[ 6.6214e-01, -2.8046e-01,  7.4298e-01],
          [-2.4776e-01,  2.3627e+00, -1.6467e-01],
          [ 5.6969e-01, -2.3224e-01,  6.5499e-01]]],
        [[[ 1.0392e+00,  7.8446e-02,  1.1745e+00],
          [-1.9321e-01, -5.8134e-01, -2.2323e-01],
          [ 7.3575e-01,  6.3480e-02,  1.2036e+00]]],
        [[[-7.4549e-01,  3.5171e-01, -7.4779e-01],
          [ 2.7267e-01, -1.0939e+00,  3.1744e-01],
          [-8.6094e-01,  1.8053e-01, -9.5986e-01]]],
        [[[-7.4761e-02, -7.4965e-02, -4.8395e-03],
          [-7.0916e-02, -9.4117e-02,  6.8479e-03],
          [-7.0671e-02, -5.7122e-02,  1.1384e-03]]],
        [[[ 2.0594e-02, -2.6332e-01, -2.0066e-01],
          [ 3.5086e-01,  2.8736e+00,  2.2054e-01],
          [ 1.2846e-01,  1.2652e-01, -2.1748e-01]]],
        [[[ 4.8172e-01,  4.0160e-01,  5.4478e-01],
          [ 4.1767e-01, -1.5756e+00,  4.4751e-01],
          [ 4.5373e-01,  4.2261e-01,  4.6070e-01]]],
        [[[-2.4105e-01,  6.2362e-02,  2.6139e-01],
          [-4.1651e-01, -1.4579e-01, -6.3829e-01],
          [-5.3446e-01, -7.7817e-01, -7.7269e-01]]],
        [[[ 9.1111e-02,  7.7725e-01,  7.9571e-01],
          [ 4.4018e-01,  1.0821e+00,  2.7830e-01],
          [ 5.1079e-01,  5.2820e-01,  4.4008e-01]]]])



cc @jerryzh168 @jianyuh @dzhulgakov @raghuramank100 @jamesr66a
</code></pre></div>