What he meant is that by calling loss.backward() you would free the computation graph. But it’s correct not to do that during validation. The reference to the loss object keeps the whole graph alive, because backward could be called.