Thanks， I think I has solve this problem by follows:
178     test(net.module, epoch, device, args.gpus)
179     qat_model = deepcopy(net)
180     qat_model.eval().to('cpu')
181     torch.quantization.convert(qat_model, inplace=True)
182     test(qat_model.module, epoch, 'cpu', args.gpus)
183     scheduler.step()

I need to run net.module first, then the qat_model in cpu is correct