“torch::NoGradGuard no_grad;” is used to prevent gradient calculation from building up memory during inference.
I am running a large model where the increase of memory is such that the memory crash unless NoGradGuard is used. I built two docker images on the same docker host server where both images are identical (same install, same make file, same cpp files…) except for the container os, i.e. one image was built on ubuntu 16.04, the other on ubuntu 18.04. Memory runs wild in the ubuntu 18.04 container while it is under control in ubuntu 16.04 container. Analysis showed that “torch::NoGradGuard no_grad;” prevents any memory increase in 16.04 but does not has any impact in 18.04.
Any explanation would be welcome?