Hi @thyeros, you can use the QAT model after prepare and before convert to evaluate in fp32 emulating int8.  It will model the quantized numerics in fp32 with the FakeQuantize modules, and it works on CUDA.  Here is an example from torchvision: https://github.com/pytorch/vision/blob/master/references/classification/train_quantization.py#L134 6