Solved by jerryzh168 in post #2 
              
              
                you’ll need to use different quant stub module for each usage because it is recording tensor statistics at that point. 
e.g. 
x = self.quant1(x)
...
x = self.dequant(x)
...
x = self.quant2(x)
...
x = self.dequant(x)