jiacheng1gujiaxin:

Can nvidia-apex only be used for inference?


No, you can use apex for training and inference.



 jiacheng1gujiaxin:

I want to deploy my model with mix precision, but I find that when I run it, my memory is not reduced.
futhermore, when i run fp16 inference by calling model.half(), the memory is not reduced either


If you are checking the used memory via nvidia-smi, note that you might see the cached memory as well. torch.cuda.memory_allocated() might give you a better number.