Hello. I have only single GPU and a very heavy model, so I am running out of memory…
Recently, I figured out that there is mixed precision training which saves the GPU memory during training.
Is NVIDIA apex an older tool to use mixed training?
I read some post and saw your comment there is a pytorch implementation which is called torch.cuda.amp.
How do I use it and what is nightly builds you commented above?