Are these per channel scales determined for quantizing the network weights or for the tensors (layer outputs)?
If it’s the first case, then can’t we compute INT8 weights offline (using the obtained scales) and upload them to the hardware without the need for having FP support? I only use the hardware for the inference phase.
If these scales are required for quantizing the internal tensors (layers outputs/inputs), then doesn’t it mean that they have to be dequantized first in each stage to be quantized again with a different scale?