How are you quantizing? Did you setup qconfigs for FloatFunctionals (q_add0, q_add1 and q_add2) as well? If you set up qconfigs for these, they will get converted to quantized::add and this op will work on quantized tensor.