I don’t fully understand the use case.
Based on this statement:



 tueboesen:

I need the gradient information of my output with regards to one of my input variables


You are calculating the gradients during validation, while



 tueboesen:

How do I free the memory in a similar fashion but without actually running loss.backward()


claims that you are never using the backward operation. How are these gradients calculated?
Note that you could also freeze all parameters, which shouldn’t get gradients and which should thus also save memory.