Hey,
Each “Node” in the graph implements a special functions to release all the resources it can here: pytorch/function.h at a46d56f988505547b0779838a022970b79123b3c · pytorch/pytorch · GitHub 2
This is called from the execution engine when retain_graph=False here: pytorch/engine.cpp at a46d56f988505547b0779838a022970b79123b3c · pytorch/pytorch · GitHub 3