hi @raghuramank100 @hx89 , I am also one of the authors of INT8 training paper on CVPR2020.
Firstly, I want to clarify that we are focus on using INT8 computation to speed up training process , not for quantization aware training. It means that we need to quantize gradient on convolution backward not only for forward.
we are now achieve nearly no accuracy (< 1% TOP1) decent on ImageNet of ResNet / MobileNet/ Inception/ ShuffleNet, Even on detection task on Pascal VOC and COCO with RetinaNet and FasterRCNN we got only ~1% mAP drop. We can check accuracy table on paper:
(see table 7,8 and 9)
And we also considering about INT8 computation implement and overhead reducing (see Section 3.6. General Purpose Training Framework)
We implement with DP4A on GTX1080TI and finally get 1.6x(forward) and 1.9x(backward) speed on convolution over cuDNN:
(see Figure 8.)
We are now plan INT8 Training V2 about quantization all CNN layer include ReLU, Pooling, Add. But We found pytorch not yet support INT8 gradient backward, So we need Pytorch team to give some help.
Feel great to see your response!