Is there other possibilities like having INT8/INT32 scale factors or having INT32 but non-per channel quantization?
The per-channel quantization is great regarding the accuracy, but I do not need that much of accuracy and instead need to perform all the calculations in INT8 or even INT32.