Solved by supriyar in post #2 
              
              
                There are currently some issues with nn.DataParallel and Quantization Aware Training. There is a WIP PR to fix it - https://github.com/pytorch/pytorch/pull/37032 
You can follow the toy example here to make sure you’re following the steps for QAT correctly https://gist.github.com/vkuzo/78b06c01f23f9…