Vasiliy_Kuznetsov:

If you are still seeing different results after transferring the weights, there could be other differences. Some things to debug would be:

are the other parameters you need to transfer (conv bias, etc)
is the input data coming in exactly the same (you are modeling quant/dequant correctly, etc)



Hi Both,
I am thankful for your time to look into the issue.
I totally agree with you both on the logic. I modified the program from @Vasiliy_Kuznetsov to demonstrate what I am trying to achieve. Would this be explained, I am thankful for that, since this is an essential step to convert pytorch model to C model:
import torch
import torch.nn as nn
# toy model

class M(nn.Module):
    def __init__(self):
        super().__init__()
        self.quant = torch.quantization.QuantStub()
        self.conv = nn.Conv2d(1,1,kernel_size=2,stride=2,padding=0,bias=False)
        self.dequant = torch.quantization.DeQuantStub()
    
    def forward(self, x):
        x_quant = self.quant(x)
        x_conv = self.conv(x_quant)
        y = self.dequant(x_conv)
        return x_quant, x_conv, y

class M_int(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv = nn.Conv2d(1,1,kernel_size=2,stride=2,padding=0,bias=False)
    
    def forward(self, x):
        # get x_quant as input
        x_conv = self.conv(x)
        return x_conv


m1 = M()
m2 = M()

def static_quant(m):
    m.qconfig = torch.quantization.get_default_qconfig('fbgemm')
    torch.quantization.prepare(m, inplace=True)
    # toy calibration
    data = torch.rand(4, 1, 2, 2)
    m(data)
    torch.quantization.convert(m, inplace=True)
    
static_quant(m1)
static_quant(m2)
# m1 and m2 now have different weights, because of different
# initialization, and different calibration data

# verify that same inputs do not lead to same outputs
data = torch.rand(4, 1, 2, 2)
print('outputs match', torch.allclose(m1(data)[2], m2(data)[2]))

# set m2's weights to be equal to m1's weights
m2.quant.load_state_dict(m1.quant.state_dict())
m2.conv.load_state_dict(m1.conv.state_dict())

# verify that same inputs lead to same outputs
data = torch.rand(4, 1, 2, 2)
print('outputs match', torch.allclose(m1(data)[2], m2(data)[2]))

m3 = M_int()
with torch.no_grad():
    m3.conv.weight.data = m1.conv.state_dict()['weight'].int_repr().type(torch.ByteTensor)
    m3.eval()
    data = torch.rand(4, 1, 2, 2)
    x_quant, x_conv, y = m1(data)
    x_conv3 = m3(x_quant.int_repr().type(torch.ByteTensor))
print('weight match', torch.allclose(m1.conv.state_dict()['weight'].int_repr().type(torch.ByteTensor), m3.conv.weight.data))
print('outputs match', torch.allclose(x_conv.int_repr(), x_conv3))


M_int model is the fresh model with integer weight loaded-in.
I expect to have the result after conv layer to be the same for m1 and m3.
I changed from linear to conv just because I am debugging for convolution2D currently.