I am running a densenet161 image classifier on gpu.
After continuous forward passes(basically 2-5 batches of images, batch_size=128), the memory gets accumulated and doesn’t get freed. Can someone tell me what is taking up so much memory?
Btw, I am using torch.no_grad() during forward pass.
@ptrblck It would also be helpful if you can tell the factors which determine memory usage during inference.