Hello pytorch community,
(Libtorch c++ 1.4 version on mac and linux):
I am currently facing the issue in my code, that iterative calls from my monte carlo tree search (building alphazero structure) to a neural network’s forward function accumulates massive amounts of memory although I have put NoGradGuard there.
The forward method in question looks like this:
std::tuple< torch::Tensor, double > NetworkWrapper::evaluate(
   const torch::Tensor &board_tensor)
{
   // We dont want gradient updates here, so we need the NoGradGuard
   torch::NoGradGuard no_grad;
   m_network->eval();

   auto [pi_tensor, v_tensor] = m_network->forward(board_tensor.to(GLOBAL_DEVICE::get_device()));

   return std::make_tuple(pi_tensor.detach(), v_tensor.template item< double >());
}

This forward method will evaluate the tensor using a convolutional net and a fully connected one.
The board_tensor also has requires_grad set to false. There is also no gradient on the tensor after the forward method, so I am unsure what all this extra memory is used for.
The tree search which calls this evaluate method all the time will eat through 32 gb memory in no time. However if I put into my main:
torch::autograd::AutoGradMode guard(false);

then there is absolutely no excessive memory usage anymore.
What exactly does AutoGradMode control, that could lead to the memory excess?