Unfortunately, Caffe2 Int8Conv doesn’t support per-channel quantization. The DNNLOWP engine that uses FBGEMM backend does support group-wise quantization if that helps you. Please see https://github.com/pytorch/pytorch/blob/master/caffe2/quantization/server/conv_groupwise_dnnlowp_op_test.py 1 for example of using group-wise quantization.