Can nvidia-apex only be used for inference? I want to deploy my model with mix precision, but I find that when I run it, my memory is not reduced.
futhermore, when i run fp16 inference by calling model.half(), the memory is not reduced either