According to this official doc 2,
" forward() - the code that performs the operation. It can take as many arguments as you want, with some of them being optional, if you specify the default values. All kinds of Python objects are accepted here. Tensor arguments that track history (i.e., with requires_grad=True ) will be converted to ones that don’t track history before the call, and their use will be registered in the graph."
In the forward call, the gradient flag doesn’t need to be explicitly turned off. So I don’t think the grad flag is the problem.